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ABSTRACT 

Since the Literacy in Libraries Across America (LILAA) 
initiative began in 1996, participation in the program’s services had fallen 
short of the amount that research indicates is needed to improve literacy 
levels substantially. Efforts to improve student persistence and implement 
service enhancements were therefore initiated in 2000. A study of students' 
persistence in 5 of LILAA' s 15 participating library literacy programs was 
undertaken around the same time. Key findings and conclusions of the 
persistence study were as follows: (1) support services (child care, 

transportation) proved the most difficult for the library literacy programs 
to implement; (2) students expressed two types of learning goals--specific 
"instrumental" goals that must be reached to realize longer-term aspirations 
and broader "transformational" goals that entail major life changes and 
libraries must acknowledge both types of student goals and build on them to 
motivate long-term participation; (3) students benefit from having several 
different type of sponsors/mentors (personal, intermediate, official) ; and 
(4) library literacy programs must preserve their caring, respectful, and 
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personalized atmosphere while simultaneously emphasizing more intensive 
participation. (Twenty-two tables/figures are included. The following items 
are appended: descriptions of five libraries studied and the study samples; a 
discussion of issues related to capturing student persistence using 
attendance record data; and an overview of the persistence study* s 
achievement study component. The bibliography lists 14 references.) (MN) 
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Overview 



Research has shown that meaningful improvements in adult literacy require more than the annual 
average of about 70 hours that adult learners spend in organized literacy instruction. Low student 
persistence is therefore a critical issue for library literacy programs and other providers of adult edu- 
cation. In response, the Wallace-Reader’s Digest Funds launched the Literacy in Libraries Across 
America (LILAA) initiative in 1996. Of the fifteen participating libraries, five are the focus of the 
LILAA persistence study, which is being conducted by the Manpower Demonstration Research 
Corporation and the National Center for the Study of Adult Learning and Literacy. This interim re- 
port fi'om the study examines the period, starting in 2000, during which the programs developed 
plans to improve student persistence and began to implement service enhancements. 

The Adult Learners and Their Initial Patterns of Participation 

• As the initiative began, literacy test scores averaged near the third-grade level on assessments of 
reading, phonetic decoding, and comprehension and at the fifth-grade level on a vocabulary test. 
Native English speakers had lower average scores than those who were learning English. 

• Participation in program services fell short of the amount that research indicates is needed to 
improve literacy levels substantially. Only two-thirds of entrants remained in the program after 
three months, and entrants averaged 57 hours of participation over 18 months. 

Early Implementation of Persistence Innovations 

• LILAA’ s continued focus on improving student persistence shaped program behavior and 
stimulated a search for ways to acWeve this goal, which had more significance than many of the 
innovations originally planned. 

• Most of the literacy programs focused on improving instruction and on changing operational 
procedures, such as expanding program hours. 

• Support services (such as child care and transportation assistance) were the most difficult inno- 
vation for the library literacy programs to implement. 

• People who entered the programs in 2001 participated for more hours during their period of ac- 
tive participation than did their counterparts in 2000. But the length of active participation did 
not differ between the two years. 

Early Implementation Lessons 

• Students expressed two types of learning goals: specific “instrumental” goals that must be 
reached in order to realize longer-term aspirations and broader “transformational” goals that en- 
tail major life changes, such as taking on a new social or work role. Literacy programs need to 
acknowledge and build on each type of goal to motivate long-term participation. 

• Learners benefited fi'om different types of sponsors — individuals who provided continuing 
encouragement and support. Library literacy programs could help students identify people who 
can play these roles, and they could support sponsors’ efforts. 

• Learners see library literacy programs as caring and respectful and, hence, as different fi'om 
other educational or social service organizations. Library literacy programs need to preserve this 
personalized atmosphere while simultaneously emphasizing more intensive participation. 

A final report, scheduled for the fall of 2003, will discuss key factors supporting and inhibiting 
participation in adult literacy programs, changes that the sites made to support student persistence 
more effectively, changes in student persistence as these reforms took effect, innovations that appear 
to be especially promising, and the relationship between participation in library literacy services and 
improved literacy skills. The final report will also explore how library literacy programs fit within 
the broader adult education system. 




-m- 



6 



Contents 



Overview 

List of Tables and Figures 
Preface 

Acknowledgments 
Executive Summary 

1 Introduction 

Participation in Library Literacy Programs 
The LILAA Persistence Study 

2 Deflning the Challenges of Improving Student Persistence 

Who Are the Program Participants? 

What Were the Early Patterns of Persistence Among LILAA Participants? 

How Did the Broader Context of Participants’ Lives Affect Their Persistence? 
Conclusion 

3 What Are the LILAA Programs Doing to Increase Persistence? 

What Kinds of Strategies Do the LILAA Programs Use? 

Information-Gathering Strategies 
Instructional Strategies 
Operational Strategies 
Support Strategies 
Conclusion 

4 What Are the Early Lessons from the LILAA Persistence Study? 

Personal Goals 

Sponsors and Sponsorship 

Respect and Care for Adult Students 

Lessons for Other Libraries 

What’s Next in the LILAA Persistence Study? 

Appendixes 

A; The Five Libraries Participating in the LILAA Persistence Study 
B: Samples Used in the LILAA Persistence Study 

C: Issues Related to Capturing Student Persistence Using Attendance 

Record Data 

D: The Achievement Study Component of the LILAA Persistence Study 

References 

Recent Publications on MDRC Projects 




7 



iii 

vii 

ix 

xi 

ES-1 

1 

1 

1 

4 

4 

9 

38 

43 

44 

44 

46 

50 

54 

55 

56 

57 

57 

60 

64 

66 

66 



69 

75 

77 

81 

93 

95 



-V- 



List of Tables and Figures 



Table 






ES.l 


The Five Libraries Participating in the LILAA Persistence Study 


ES-2 


2.1 


Demographic Characteristics of All Program Participants, January 2000 Through 
December 2001 


6 


2.2 


Participation Patterns in the First 18 Months for Program Entrants in the Extended 
Follow-Up Sample 


16 


2.3 


Total and Average Monthly Hours of Participation in the First 18 Months for Pro- 
gram Entrants in the Extended Follow-Up Sample 


20 


2.4 


Total and Average Monthly Hours of Participation in the First 18 Months for Pro- 
gram Entrants in the Extended Follow-Up Sample, by Subgroup 


32 


2.5 


Participation Patterns in the First 18 Months for Program Entrants in the Extended 
Follow-Up Sample, by Program 


36 


2.6 


Total and Average Monthly Hours of Participation in the First 18 Months for Pro- 
gram Entrants in the Extended Follow-Up Sample, by Program 


37 


B.l 


Samples Used in the LILAA Persistence Study 


76 


D.l 


Performance Levels on the Basic English Skills Test (BEST) for English Lan- 
guage Learners 


85 


Figure 






2.1 


Literacy Test Age/Grade Equivalents 


8 


2.2 


Literacy Test Age/Grade Equivalents for English Speakers and English Learners 


10 


2.3 


Literacy Survey Responses: “What do you find hard about reading?” 


11 


2.4 


Literacy Survey Responses: “What do you find easier now than before you en- 
tered this program?” 


12 


2.5 


Percentage Participating Over Time for Program Entrants in the Extended Follow- 
Up Sample 


14 


2.6 


Percentage Participating Over Time for Program Entrants in the Short Follow-Up 
Sample, by Quarter of Program Entry 


18 


2.7 


Average Monthly Hours of Participation Over Time for Program Entrants in the 
Short Follow-Up Sample, by Quarter of Program Entry 


22 


2.8 


Percentage Participating Over Time for Program Entrants in the Extended Follow- 
Up Sample, by Gender 


24 


2.9 


Percentage Participating Over Time for Program Entrants in the Extended Follow- 
Up Sample, by Age Group 


26 


2.10 


Percentage Participating Over Time for Program Entrants in the Extended Follow- 
Up Sample, by Employment Status 


27 


2.11 


Percentage Participating Over Time for Program Entrants in the Extended Follow- 
Up Sample, by Ethnicity 


28 


2.12 


Percentage Participating in Any Activity Over Time for Program Entrants in the 
Extended Follow-Up Sample, by Primary Activity 


30 


2.13 


Percentage Participating Over Time for Program Entrants in the Extended Follow- 
Up Sample, by Program 


34 




-vn- 



8 



Preface 



As the U.S. economy has changed, an increasing number of jobs have come to demand 
English literacy and communication skills. At the same time, millions of American adults can- 
not reap the rewards of literacy because their prior education was inadequate, they have learning 
problems, or they are in the process of learning English. 

Many adult learners aiming to improve their language skills enroll in programs that 
provide literacy instruction, but research has shown that adult students generally do not partici- 
pate long enough or intensely enough to achieve significant gains. In recent years, government 
funders of adult literacy programs have sought to change this pattern by requiring the programs 
that they fund to demonstrate positive effects on students’ literacy skills. 

Library-based programs are an important part of the network of organizations that serve 
adult literacy students. Libraries exist in most communities, are open to all, and provide a more 
inviting setting for adult learning than do other education institutions. Building on these natural 
advantages, the Wallace-Reader’s Digest Funds launched the Literacy in Libraries Across Amer- 
ica (LHAA) initiative to help 15 library literacy programs around the country develop and im- 
plement new ways to increase adult learners’ persistence and to test whether the new strategies 
lead to longer, more intense program participation and to improvements in literacy. 

Five of the LILAA programs are the focus of the LILAA persistence study, which is be- 
ing conducted for the Wallace-Reader’s Digest Funds by the Manpower Demonstration Re- 
search Corporation in collaboration with the National Center for the Study of Adult Learning 
and Literacy. Given the well-established link between substantial program participation and 
literacy gains, this research takes a logical next step by focusing on efforts to improve adult 
learners’ persistence. And by developing new ways to measure and analyze participation and 
identifying and fielding achievement tests suitable for adult literacy students, the study offers 
lessons and insights for other programs that seek to measure gains in participants’ literacy skills. 

This report presents interim findings fi'om the LILAA persistence study that can help 
policymakers and program managers improve adult literacy services. It examines baseline par- 
ticipation levels, explores the experiences of different types of adult learners, describes the vari- 
ety of innovations being tested in the LILAA programs, and discusses how to build strong rela- 
tionships with adult learners as they begin to participate in a literacy program. A final report, 
scheduled for publication in the fall of 2003, will provide a fuller picture of the innovations that 
the LILAA programs implemented, examine how student participation changed over time, ana- 
lyze whether and how literacy levels improved, and explore the implications of the findings for 
the broader adult education and literacy system. 



Kent McGuire 
Senior Vice President 
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Executive Summary 



Weak literacy skills can mean the difference between holding a job and being unemployed, 
can limit career choices, and can prevent people from participating in the civic life of their commu- 
nity. Research shows that meaningful improvements in adult literacy require a “threshold” level of 
participation in an adult literacy program; sporadic participation may well make little difference in 
literacy. Unfortunately, however, in a one-year period, most adult literacy students spend fewer 
than 70 hours engaged in organized literacy insfruction. Given their low initial literacy levels, many 
would need more hours of insfruction to make progress and several years of study to accomplish 
their literacy goals. 

Many public libraries in the United States provide literacy services, and some provide di- 
rect insfruction through individual tutoring, classes or small groups, and computer-assisted learning 
— often to students who have no other education options because of their low literacy skills. Con- 
cerns about low levels of student persistence have become a major policy and program issue for 
library literacy programs and other providers of adult education, especially as more federal funding 
has become contingent on showing student progress. 

With a goal of addressing problems of student persistence, the Wallace-Reader’s Digest 
Funds launched the Literacy in Libraries Across America (LILAA) initiative in 1996 and con- 
tracted with the Manpower Demonstration Research Corporation and the National Center for the 
Study of Adult Learning and Literacy to study the effort. Of the fifteen libraries participating in the 
initiative, five are the focus of the LELAA persistence study. Each of these five libraries and their 
nine branch programs (described in Table ES.l) developed plans to improve student persistence, 
and they received funding from the Wallace-Reader’s Digest Funds to implement service en- 
hancements in 2000. This report describes the adult learners who are served by the programs, ana- 
lyzes initial patterns of learner persistence (in terms of both length and intensity) prior to the full 
implementation of program enhancements, and documents the early progress of these programs in 
implementing innovations to increase participation. Because this is the first study of its kind, the 
findings break new ground in many respects. 

Key Findings 

The Adult Learners and Their Initial Patterns of Participation 

• Overall, the literacy levels of students in the LILAA programs were low, 
and native English speakers, on average, achieved at lower levels than those 
who were learning English. 
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The Five Libraries Participating in the LILAA Persistence Study 
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NOTE: ^Quantitative data were collected only from the Glenwood program. 



At the start of this study, 250 students took literacy tests, and a follow-up test was 
scheduled for one year later. Average initial scores ranged from near the third-grade level on 
assessments of reading, phonetic decoding, and comprehension to the fifth-grade level on a test 
of vocabulary. Average scores for students who were learning English exceeded average scores 
for native English speakers who were participating in adult basic education (ABE) programs, by 
one to three grade levels, depending on the test used. This difference may have arisen in part 
because the students learning English may have had stronger literacy skills in their native lan- 
guage. In any case, these low initial levels of literacy suggest that students need to participate in 
many hours of instmction over an extended period in order to achieve their literacy goals. 

• Adults participating in the LILAA programs all share one thing in com- 
mon — a desire to improve their literacy skills — but in some other 
ways they are a diverse group, reflecting the variety of the communities 
served. 

The clientele of library literacy programs very much reflect the characteristics of their 
communities as well as local perceptions of the library programs and their services. The adult 
learners in these programs were almost entirely people of color (less than 5 percent were white), 
with each site drawing half or more of its students fiom a single racial or ethnic group. All the 
programs attracted students of a broad age range. Slightly more than one-third of students (36 per- 
cent) were between ages 21 and 35, and slightly more than one-third (36 percent) were between 
ages 36 and 50. Eight percent were younger than 21, and a full 20 percent were older than 50. 

• Prior to the implementation of program innovations, the length and in- 
tensity of students’ participation in services fell short of the amount 
needed to improve literacy levels substantially. 

Prior to the full implementation of innovations to support student persistence, only two- 
thirds of the adults entering the programs participated at any level in the third month of follow- 
up. After this rapid early drop-off in participation, the participation rate continued to decline, 
gradually dropping to 28 percent in the twelfth month of follow-up and to 1 5 percent in the 
eighteenth month. Entrants averaged 57 hours of participation over 18 months, though in 
months when students were active, they averaged 7 hours of participation per month, or less 
than 2 hours per week. 

There do not seem to be major systematic differences among learners by gender, but 
students over 36 years of age and those employed full time at entry tended to participate more. 
There are some differences across racial and ethnic groups, but these are intertwined with dif- 
ferences related to local program characteristics and communities, because groups tend to be 
clustered in specific sites. There are major differences in participation according to learners’ 
primary mode of instruction; those primarily working with an individual tutor participated the 
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most, and those studying primarily in classroom settings participated the least. However, this 
effect could also be driven by local program differences (sites tended to emphasize a particular 
mode of instmction) or by differences in the characteristics of participants in each mode (be- 
cause different students were attracted to different modes of learning). 

• Movements in and out of the program were frequent 

Many students who stopped participating did so abruptly (without a gradual decline in 
hours), and about one-third of those who stopped participating for a month returned to the pro- 
gram in some later month. Maintaining contact with a program’s currently inactive learners 
could be an important support to student persistence. 

• Students in these library literacy programs used the services in four dif- 
ferent ways during the early phase of the LILAA initiative, suggesting 
that a range of programmatic responses may be needed to address low 
participation. 

Some adult literacy students are intermittent participants who join and leave a program 
several times. Breaks in participation are often caused by a personal crisis, and students return 
when the problem has passed. Programs should help these students maintain a connection dur- 
ing periods of inactivity and should welcome them back when they are able to resume participa- 
tion. Short-term students participate intensely but briefly to accomplish a specific goal, such as 
admission to a training program or preparation for a citizenship test. Programs should help such 
learners identify long-term goals and other programs that they can use. Long-term students are 
active in programs for an extended period, often participating in computer-assisted instmction 
or self-study activities as well as tutoring and classes. For some, specific, realistic literacy goals 
can be developed, but others participate because it provides social interaction and a sense of 
community. Mandatory participants are required to attend a program as a condition of receiving 
some type of aid or to comply with conditions of parole. If these learners can identify goals of 
their own — related to but separate fiom the agency that referred them — this could help moti- 
vate them to participate more substantially. 

• Many adult learners face serious difficulties in other aspects of their 
lives, and these often hinder their efforts to participate steadily and in- 
tensely in literacy services. 

The adult learners in library literacy programs are almost entirely fi-om low-income 
households, and a substantial minority may have undiagnosed learning disabilities that hamper 
their progress in the literacy program and in other aspects of their lives. They may also have 
health problems or histories of substance abuse that have prevented them fi-om achieving steady 
employment. Those who are learning English may be relatively new to the United States, so- 
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cially isolated, and struggling to acclimate themselves to a new culture. Some have been in abu- 
sive family relationships and have not received emotional support in their efforts to improve 
their literacy skills. 

These patterns of participation illustrate the importance of the LILAA initiative and the 
need to create broad-based interventions that extend and deepen engagement in program liter- 
acy activities. Given most adult learners’ low levels of literacy, they will make only slow pro- 
gress in meeting their literacy goals with the amount of participation that was reported in the 
early stages of the LILAA initiative. 



Early Implementation of Persistence Innovations 

The LILAA persistence study involves a focus on student persistence and the imple- 
mentation of specific innovations in four categories: information gathering to learn more about 
students and developments in the adult literacy field, instructional improvements to make in- 
stmctional techniques more engaging and useful, operational changes to reform program pro- 
cedures and make the services more accessible, and support services to help students overcome 
personal and social barriers to persistence. 

• In the initial implementation phase, the LILAA initiative’s continued fo- 
cus on improving student persistence shaped local programs’ behavior, 
and the resulting search for ways to achieve this goal had more signifi- 
cance than many of the specific innovations that were originally 
planned. 

During the initial implementation phase, fi"om January 2000 to June 2001, some 
planned interventions were dropped or delayed, while others moved ahead and new ones 
emerged. The constant during this period was the emphasis within the LILAA project on in- 
creased student persistence, both as a goal to be sought and as an operational indicator of stu- 
dent progress to be tracked. 

• Information that the programs collected at the start of the initiative con- 
cerning students’ needs and perspectives and alternative program ap- 
proaches did not lead to major changes in local plans for reform; most 
sites focused on their existing plans for innovations. 

As part of the LILAA persistence study, programs were expected to collect information 
fi-om participants about their needs, the barriers they faced to more intensive participation, and 
their ideas about how to improve program services. The findings fi'om this data collection effort 
— following the model of action research — were to be used in the design of subsequent inter- 
ventions. In most cases, the information gathering was done more slowly than originally envi- 
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sioned. Programs typically began to move forward with new interventions before completing 
efforts to fully document and formally analyze the participants’ perspectives on possible im- 
provements, though, in some cases, information was used to justify changes already under way 
or to refine and add new strategies. 

• Throughout the initial implementation period, the literacy programs 
focused mostly on improving instruction and changing operational 
procedures. 

Instructional innovations included improved student orientations and tutor training, new 
ways to build more effectively on learners’ goals, new methods of delivering literacy services 
(for example, through enhanced computer labs or new methods of group instruction), and 
clearer standards for instruction to ensure that each student received services meeting a mini- 
mum threshold of adequacy. Operational strategies involved changing and expanding program 
hours. These enhancements aimed to improve the direct services in library literacy programs. 

• Support services were the most difficult innovation for the library liter- 
acy programs to implement. 

Some adult learners arrived at the literacy program needing multiple support services. 
Because library literacy programs could not directly offer all the needed supports (such as child 
care, counseling, health services, transportation assistance) even if they wished to, efforts to en- 
hance such offerings were dependent on the cooperation of other agencies, many of which were 
already hard-pressed to meet the demand for their services. In addition, some libraries were 
hesitant to take on the responsibility of addressing the support needs of adult learners, out of 
concern that this could be a “slippery slope” pulling them into the role of a social service pro- 
vider and diverting them from the core mission of improving literacy. Their reluctance was also 
connected to the long-term costs of such services and to worries about not being able to con- 
tinue services to students in need. Finally, some library staff members were concerned about the 
equity of providing special support services to learners in literacy programs when other library 
patrons might have as great a need for such assistance. 

• Data collection and reporting on students’ characteristics and participa- 
tion improved over time, though the problems encountered may herald 
difficulties to be faced as federal reporting requirements increase. 

At the outset of the persistence study, the library literacy programs had varied experi- 
ences in data collection; some were able to meet the project’s requirements with only modest 
adjustments, while others needed substantial help in making changes. As part of the initial im- 
plementation phase, the programs built new data collection systems or enhanced existing ones 
and were able to provide information about learners’ background characteristics and hours of 
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participation. However, these improvements required new commitments of staff time for data 
collection and reporting, which in this case were supported by special project funding. In addi- 
tion, the data collection effort raised concerns about confidentiality, because some staff saw it as 
contrary to traditional library values, which hold that patrons can use services as they wish, 
without fear that anyone is monitoring their activities. 

• As the program innovations begin to take hold, there are some signs of 
an encouraging trend in the intensity of participation among students 
who are active in the programs. 

It is premature to judge whether the program interventions will increase learner persis- 
tence, but there are early suggestions of encouraging trends in the intensity of participation 
among active students. Those who entered a library literacy program in 2001 participated for 
more hours while active than their counterparts who entered in 2000. However, the percentage 
of students who remained active over time did not increase. Further analysis of these trends will 
be a focus of the final report. 



Early Implementation Lessons 

At this early stage in program implementation, the lessons from the LILAA persistence 
study center on key challenges in developing a strong initial connection with adult learners and 
on promising approaches that are suggested by the initial year’s experience. 

• Adult learners expressed two types of goals, and literacy programs need 
to find ways to acknowledge and build on each of these. 

Many literacy students may have specific goals, such as passing the General Educa- 
tional Development (GED) examination or an occupational test, reading job-related material 
more quickly and accurately, or reading stories to a young child. Such instrumental goals can 
motivate many adult learners to participate in a literacy program. But some students may also 
express broader goals that may initially strike program staff as unfocused or unrealistic. Under- 
lying these transformational goals may be a profound desire to improve the conditions of one’s 
life, to expand one’s repertoire of skills, or to change one’s social identity. Aspirations like these 
can also be a motivator of long-term participation in a program, especially if they can be linked 
to intermediate objectives. Literacy programs should seek ways to tap into such aspirations, and 
initial efforts at goal-setting should not be restricted in ways that might exclude or devalue these 
more nebulous but still-important student goals. 

• Learners benefited from different types of sponsors — individuals who 
provided continuing encouragement and support Literacy programs 
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could help students identify people who can play these roles and could 
support sponsors’ efforts. 

Because most adult learners face many challenges outside the literacy program, problems 
may interrupt participation, and disappointments may undermine motivation. Early experience in 
the persistence study suggests that students can benefit from different types of sponsors. Personal 
sponsors are often family members or close fiiends who have a long-term relationship with a stu- 
dent and can provide continuing emotional, literacy, and informational support. Intermediate 
sponsors typically play a supportive role through a student’s participation in a religious, social, 
self-help, or educational institution. Official sponsors are professionals such as social workers, 
welfare case managers, literacy staff members, or parole officers who are providing intermittent, 
targeted institutional support to students. Library literacy programs can assist students in identify- 
ing individuals and organizations to serve in sponsoring roles and can aid sponsors in their efforts. 

• Learners see library literacy programs as caring and respectful and, 
hence, as different from other educational or social service organizations 
with which they are involved. Library literacy programs need to pre- 
serve this personalized atmosphere while simultaneously emphasizing 
more intensive participation. 

Directors of the LILAA programs reported that many students were demoralized by the 
feeling that schools or other program providers did not treat them with respect or strive to meet 
their needs. In response, and noting that many students are members of minority groups, direc- 
tors have attempted to create a climate that — unlike mainstream institutions — reflects stu- 
dents’ cultural experiences and values. Many students report that libraries feel different, offering 
them a caring and respectful setting in which to improve their literacy skills and express them- 
selves. At the same time, the LILAA initiative and the early experience in the first program year 
support the need to strengthen instruction, provide cultural diversity training to tutors, and build 
a community that is open and communicative. Libraries should be careful to preserve their ex- 
isting atmosphere as they work to strengthen these aspects of literacy programs. 



Implications of the Findings for the Project 

This initial research has also highlighted several key themes that cut across the literacy 
programs in the LILAA persistence study. At times in this report, findings challenge both the 
stereotypical views of what “all” literacy programs and students look like and the contention 
that all programs are different because of unique local circumstances. Future research will con- 
tinue to explore both the diversity of programs and people and the patterns of circumstances and 
behavior. In addition, the research will continue to explore whether and how library literacy 
programs have come to understand the attractions of participation for students and the barriers 



they face and have crafted effective strategies to increase the benefits and reduce the costs of 
participation. 

The findings of this interim report set the stage for further analysis to be presented in a 
final report scheduled for the fall of 2003. That report will focus on five questions: 

1. What are the key factors supporting and inhibiting participation in adult lit- 
eracy programs? What are the implications of these factors for the design of 
strategies to improve learner persistence? 

2. Over time, how did the sites in the persistence study change their operations 
to support student persistence more effectively? What innovations were 
strengthened or put in place during the second and third year of the initiative? 

What operational lessons emerged as programs worked to support improved 
persistence? 

3. As the reforms took effect, did student persistence improve over time? Did 
learners who entered the LILAA programs during this later period persist 
longer or participate more intensely? Were there differences among types of 
students? Were there differences across the sites? 

4. Are there types of innovations that appear to be especially promising as ways 
to improve student persistence? What efforts and resources were needed to 
put these innovations in place? What are the implications for future program 
design and operations? 

5. What is the relationship between participation in library literacy services 
and improved literacy skills? What were the gains in literacy achievement 
test scores between the initial testing of students and a follow-up test? Did 
students who participated more in services show greater gains? Were there 
types of students who showed especially strong gains? Who showed little or 
no gains? What are the implications for program design? 

The upcoming research will also explore how library literacy programs fit within the broader 
adult education system. 

Each of these questions breaks new ground for program operations and research on 
adult literacy. The answers will help program operators and funders more effectively serve 
adults who face serious barriers to full participation in the nation’s economy and civic life. 
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Chapter 1 

Introduction 



Participation in Library Literacy Programs 

The growing importance of communication and information technology in the U.S. 
economy has increased the need for literacy and math skills, even at jobs that previously re- 
quired little education or training. Weak literacy skills can mean the difference between holding 
a job and not being able to find or keep one and can prevent people from participating in their 
community’s civic life. Many public libraries in the United States provide a range of literacy 
services, often to students who have no other education options. Because libraries are perma- 
nent community institutions with diverse resources, they are in a unique position to address a 
wide range of educational and support needs of adult literacy students and their families. 

A primary reason why adult students participate in library literacy programs is to 
achieve meaningful improvements in literacy skills. But the acquisition of literacy skills is not a 
simple, predictable process, in which one extra hour of study results in a specific measurable 
gain in literacy skills. Instead, research shows that meaningful improvements in adult literacy 
require a certain “threshold” level of participation in literacy programs.' 

Unfortunately, participation in literacy programs is only one of many competing de- 
mands and activities facing adult students, and most adult students spend fewer than 70 hours 
annually in organized instruction — far short of the intensity needed to truly progress. Con- 
cerns about low levels of student persistence have become a major policy and program issue for 
library literacy programs and other providers of adult education, especially as more federal 
funding becomes contingent on showing student progress.'’ 



The LILAA Persistence Study 

The Wallace-Reader’s Digest Funds launched the Literacy in Libraries Across America 
(LILAA) initiative in 1996 to help 15 U.S. libraries improve their literacy services, and it con- 
tracted with the Manpower Demonstration Research Corporation and the National Center for 
the Study of Adult Learning arid Literacy to study the effort. As identified in Box 1.1 and more 
fully described in Appendix A, five of these libraries are the focus of the LILAA persistence 

'See, for example. Bos, Scrivener, Snipes, and Hamilton, 2002. 

^Young, Morgan, Fitzgerald, and Fleishman (1994) reported that adult student in federally supported 
programs spent an average of 58 hours aimually in organized instruction, though recent unpublished fig- 
ures suggest that this might have risen to around 70 hours. 

^See, for exanple, “Measures and Methods for the National Reporting System for Adult Education” (2001). 
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study. This is the third of four reports from the study examining strategies that library literacy 
programs have undertaken to increase learner persistence, lessons emerging from this effort, and 
results of these persistence strategies for adult students.'* The study seeks to; 

o Clearly define the concept of learner persistence 

o Develop better tools for measuring persistence 

• Foster a deeper understanding of the barriers to persistence and the supports 
that can enhance it 

• Evaluate approaches for supporting greater persistence 

One of the goals of the LILAA persistence study is to move beyond simple characteri- 
zations of library literacy programs and the students they serve to illustrate the diversity of set- 
tings, services, and students and to explore the implications of this variety. At times, discussions 
of library literacy programs imply a “sameness” across programs and people that hides both 
strengths and challenges. In contrast, this report — in its program descriptions, profiles of peo- 
ple, and analysis of participation — seeks to move beyond aggregate discussions and to present 
this research project as starting to chart a middle ground between the statements “Everyone and 
every program is the same” and “Everyone and every program is unique.” The goal is to use 
disaggregation to clarify policy and practice issues and opportunities and to identify subcatego- 
ries of program strategies and participants for which there are common themes and lessons. 

This report describes the initial experiences of library literacy programs as they 
launched new efforts to enhance learner persistence. Following this introductory chapter. Chap- 
ter 2 defines the challenges of increasing persistence by discussing the adult learners served by 
the programs in this study and their initial levels of literacy and service participation as LILAA 
programs began to introduce new strategies to increase persistence. Chapter 3 focuses on the 
early efforts of programs to introduce new strategies, discussing approaches that were success- 
fully introduced during this early period and new ideas that emerged, as well as strategies that 
— at this point — have not been put in place. Chapter 4 explores early lessons related to the 
challenges that learners face and the strategies that libraries have employed to increase persis- 
tence. The focus of Chapter 4 — given the stage of the project — is on lessons for creating and 
maintaining a strong connection with learners as they begin participating in a literacy program. 

The final report on the project, to be completed in 2003, will describe program imple- 
mentation in the second year of the initiative, examine whether learner persistence increased 



‘'For early descriptions of the initiative' and the participating libraries, see Coinings and Cuban 
(2000); Coinings, Cuban, Bos, and Taylor (2001). 
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during the period of the initiative, explore links between the strategies put in place to increase 
persistence and the observed changes, and analyze the extent to which program participation is 
linked to improved literacy (as measured by a battery of standardized literacy tests). 



Box 1.1 

The Five Libraries Participating in the LILAA Persistence Study 

Greensboro (North Carolina) Public Library’s literacy programs evolved from a commu- 
nity effort to address the needs of adults with low literacy skills. This resulted in Literacy 
2000, a collaborative plan supported by the library, the community college, Reading Connec- 
tions (a nonprofit agency supporting one-on-one tutoring), and community agencies. Literacy 
programs that started through Literacy 2000 are located at two of Greensboro Public Li- 
brary’s nine branches, Glenwood and Chavis. 

New York Public Library (NYPL) has helped immigrants assimilate into the local commu- 
nity for more than a century. Nine of the 85 branches of the NYPL provide literacy services 
through their Centers for Reading and Writing (CRWs), which target adults at the lowest lev- 
els of literacy. The CRWs still serve many new immigrants, but the focus of the New York 
City Adult Literacy Initiative (NYCALI) is currently on literacy for English-speaking adults. 
The LILAA persistence study is focused on three of the CRWs; Fordham and Wakefield in 
the Bronx and Seward Park in Manhattan. 

Oakland Public Library’s Second Start adult literacy program, started in 1984, is a small, 
community-based program with both classroom insfiuction and one-on-one tutoring. Housed 
in a downtown office building. Second Start’s multiethnic staff offer insfiuction in math, writ- 
ing, and spelling as well as a program for stress reduction. Second Start also offers a com- 
puter-assisted literacy program, with approximately 20 computers. 

Queens Borough Public Library is among the nation’s oldest and largest library systems 
and serves one of the most diverse populations. The adult literacy program, founded in 1977, 
has its roots in the earlier library-based programs that provided education to immigrants. Six 
of the library’s 62 branches support Adult Learning Centers (ALCs), which enroll more than 
2,500 adults each year, and three of these branches that have ALCs are included in the 
LILAA persistence study; Central, Flushing, and Rochdale Village. 

Redwood City (California) Public Library’s Project READ serves approximately 200 
adults who have low literacy skills. More than 180 volunteers tutor these adults in both one- 
on-one settings and small groups. Two-thirds of the students are Hispanic, and many do not 
have a high school diploma or General Educational Development (GED) certificate. Project 
READ serves a variety of other populations, including children and teens, adults in a local 
jail, and students in a Special Needs program for students with learning disabilities. 




-3- 



22 



Chapter 2 



Defining the Chaiienges of 
improving Student Persistence 



The program participants in the Literacy in Libraries Across America (LE.AA) initia- 
tive are adults from diverse backgrounds who seek to improve their literacy skills for a variety 
of reasons and who have a number of different goals. Some are recent immigrants who are liter- 
ate in their native language but want to increase their English fluency. Others do not have a high 
school diploma and want to prepare for the General Educational Development (GED) exam, 
designed to provide the equivalent of a high school credential. Still others may find that their 
low literacy level is a barrier to success in today’s job market. These students may choose li- 
brary literacy programs over other adult education programs because they cannot find other lit- 
eracy instmction that is appropriate at their low literacy levels, because they benefit from the 
wide range of services provided by the libraries, or because they prefer the libraries’ instmc- 
tional approach over the methods used in more traditional educational settings. Many of the par- 
ticipants live in the communities near where the public libraries are located. 

The challenges of improving student persistence arise out of the difficult lives of these 
adults, their existing levels of literacy and program participation, and their personal goals. This 
chapter begins by introducing the students in the nine library literacy programs during the first two 
years of the LffiAA initiative, from January 2000 through December 2001. It presents some basic 
characteristics of these adult learners, discusses their literacy levels, and sheds light on students’ 
views of their own literacy and literacy learning. Next, it presents the existing levels and patterns 
of persistence among these adult students. Drawing on attendance records from each of the library 
literacy programs, the analyses presented here are groundbreaking for the adult literacy field, pro- 
viding a reliable baseline about student persistence for the first time. The chapter concludes with 
case studies that illustrate various patterns of persistence by describing individuals’ experiences in 
the literacy programs and the life circumstances that affect their persistence. 



Who Are the Program Participants? 

Characteristics of LILAA Program Participants 

The adult learners in the LILAA programs all share one thing in common — a desire to 
improve their literacy skills — but in some other ways they are a diverse group, reflecting the 
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variety of the communities served.' As Table 2.1 shows, the programs tend to serve more 
women than men, with only Oakland having fewer women (45 percent) than men. In the other 
programs, the percentage of women ranges from 53 percent in Greensboro^ to 70 percent in 
Flushing (Queens Borough Public Library). Although the programs attract students of all ages 
(the youngest in the sample was 15, and the oldest 100), people between ages 20 and 50 pre- 
dominate. A later discussion suggests that there may be differences in the ways that younger 
and older adults use the library literacy programs. 

Students’ race and ethnicity vary substantially across the programs, largely reflecting the 
communities in which they are situated. For example, at Oakland’s Second Start program, stu- 
dents are predominantly African-American (78 percent); Redwood City’s students are predomi- 
nantly Hispanic (78 percent); and Eushing’s students are predominantly Asian (69 percent). 
Overall, 42.5 percent of the students served by the programs in the LILAA study are black (in- 
cluding African-Americans, Afro-Caribbeans, and a small number of Africans); 25.3 percent are 
Hispanic; 23.4 percent are Asian; and the remaining 8.8 percent are either white or of another race 
or ethnicity. 

Slightly more than half the students who provided employment information were not 
employed when they entered the program.^ Across the programs, the percentage who were em- 
ployed at program entry ranged from 22 percent to 63 percent, illustrating that some programs 
may face different challenges in increasing the persistence of working adults. More students 
were employed full time than part time, with the percentage of full-time employees at program 
entry ranging from 16 percent to 54 percent."' As the later discussion of persistence shows, how- 
ever, employed learners do not participate less than those without a job at program entry, sug- 
gesting they may have other characteristics that support their literacy learning. 

Literacy Levels of LILAA Program Participants 

The students at the LILAA programs tend to have very low literacy skills. For many, 
the level of their literacy skills is below that needed for other adult education options, for access 
to job training opportunities, and for many jobs. Many are not able to read to their children or to 
help with the children’s homework. And for many of these students, limited literacy skills affect 
their ability to be well-educated consumers, active citizens, and informed voters. 



'See Appendix B for a full description of the various samples used in the analyses. It is very likely 
that the programs served more students than those for whom data are available; as discussed in Appendix 
C, it is extremely challenging to accurately track every student. 

^Greensboro data were collected only from the Glenwood branch. 

"About 75 percent of the 4,000 students provided this information. The percentages reported in the 
text are for those who provided employment information. 

‘‘The programs do not update the employment status of their students after enrollment. 
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As part of the LD1,AA initiative, 250 students in five of the nine LD1,AA programs took 
literacy tests.^ While this sample is not random and includes students who have been participat- 
ing in literacy services prior to testing, the students illustrate much of the breadth of the LILAA 
adult learner population. Figure 2.1 presents students’ scores for three of the four literacy tests 
administered to adult learners, while the scores for the Basic English Skills Test (BEST) — a 
test only for non-English speakers — are discussed separately. The figure presents either a 
grade-level or an age-level equivalent score for each of the tests. Most of the students tested 
scored at very low literacy levels, whether illustrated by the distribution of their scores as shown 
in Figure 2.1 or by the score of the average student. There is substantial variability among the 
test-takers’ performance levels on these tests, as indicated by relatively high standard errors for 
all the tests and by average scores that are strongly influenced by high scores for a small per- 
centage of test-takers. 

For the Peabody Picture Vocabulary Test (PPVT), which measures vocabulary skills, 
the average score among the learners who successfully completed the test was the age-level 
equivalent of 10.4 years (approximately the fifth grade). For the Adult Basic Learning Examina- 
tion (ABLE), the average standardized score for the reading comprehension components, 3.2, 
translates to the equivalent of slightly below a fourth-grade reading level. And for the Test of 
Word Reading Efficiency (TOWRE), a test of reading rate and word recognition, the majority 
of students scored at or below the third-grade level on the two subtests, with the average student 
scoring at a grade level of 2.9 for the Sight Word Efficiency (SWE) subtest and at a grade level 
of 2.6 for the Phonemic Decoding Efficiency (PDE) subtest. All three of these tests signify very 
low levels on a variety of literacy skills. 

For those who took the BEST (students in English for Speakers of Other Languages, or 
ESOL, programs), the mean score of 61 translates to a level 6 on a scale of 10 student-ability 
levels developed by the Mainstream English Language Training (MELT) project (not shown in 
Figure 2.1). Level 6 signifies the ability to satisfy most basic survival needs, including routine 
work and social demands. It also typically means that the test-taker can follow simple oral and 
written instructions in both familiar and unfamiliar settings and can communicate with native 



^Participating students were sampled from the Central and Flushing branches of Queens Borough 
Public Library, the Fordham branch of New York Public Library, Oakland Public Library, and Redwood 
City Public Library. These programs were chosen because their instruction includes adult basic education 
(ABE) and English for Speakers of Other Languages (ESOL) and because they represent a variety of 
demographic characteristics, types of instructional formats, and student population sizes. The literacy 
levels indicated by the scores measure the abilities of students in the programs, rather than their abilities 
on entering a program; those participating in the testing have been engaged in literacy activities at a 
LILAA program for varying amounts of time. For more information about the achievement study and the 
battery of tests being used, see Appendix D. 
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The LILAA Persistence Study 
Figure 2.1 

Literacy Test Age/Grade Equivalents 




Under 5 Age 5-7 Age 8-10 Age 11-13 Age 14-16 Over 16 
PPVT Age Equivalent 




Grade 1 or Grade 2 Grade 3 Grade 4-5 Grade 6-7 Grade 8 or 
below higher 

ABLE Grade Equivalent 




Grade 1 or Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 or 



below higher 



TOWRE SWE Grade Equivalent 




Grade 1 or Grgide 2 Grade 3 Grade 4 Grade 5 Grade 6 or 
below higher 



TOWRE PDE Grade Equivalent 



SOURCE: LILAA Achievement Study Database. 

NOTES: Sample sizes varied between the tests, from 192 to 241. 

Half the respondents were between ages 34 and 52; a quarter were older, and a 
quarter were younger. 
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English speakers who are not used to dealing with those whose English is limited.® A chart ex- 
plaining the competencies associated with all 10 student-ability levels and how BEST scores 
relate to them can also be found in Appendix D. 

As shown in Figure 2.2, English language learners performed significantly higher than 
the English speakers on two of the literacy tests. This may seem surprising at first, but there are 
several likely reasons for this pattern. The initial levels of literacy among ESOL students may 
be higher because many of them already have substantial literacy skills and formal education in 
their own language.’ They may, therefore, be able to transfer their word recognition, decoding, 
and reading comprehension skills to their English learning, unlike English-speaking ABE learn- 
ers, who have not developed these basic skills. In addition, their schooling may have focused on 
reading and writing academic English, with less attention being paid to their verbal abilities. 
Finally, because the test scores already capture some of the effects of students’ program partici- 
pation, the higher scores recorded by English learners may suggest that their literacy skills in- 
crease more quickly as they participate in these programs. 

Students’ Perceptions of Their Literacy Skills and Gains 

The students who participated in the achievement study also took a short survey an- 
swering questions about how they assessed their own reading skills and improvements in liter- 
acy.* Interestingly, the students did not necessarily see themselves as having veiy low literacy 
skills, ranking themselves as 5.5, on average, on a scale of 10.® Nevertheless, as shown in Figure 
2.3, students reported difficulty in pronouncing and spelling words, understanding hard words, 
and understanding what they read overall. They do see benefits fi-om participating in a literacy 
program; as shown in Figure 2.4, they see improvements particularly in their reading compre- 
hension and in their ability to deal with difficult words. 



What Were the Early Patterns of Persistence Among LILAA 
Participants? 

For students in literacy programs to succeed, they must find activities that fit their 
schedule and address their goals, remain active and participate on a regular basis, and overcome 



^Center for Applied Linguistics, 1989. 

^In one of the LILAA programs, foreign-bom college students and their spouses make up a signifi- 
cant portion of the adult learners. 

This survey, developed by the team of qualitative researchers at the National Center for the Study of 
Adult Learning and Literacy, drew and modified some questions from the “Interviewer Questionnaire from the 
Adult Reading Con^onent Study,” by John Strucker and Rosalind Davidson (NCSALL, Forthcoming, 2003). 

^This is consistent with findings nationally that there is a discrepancy between self-perception of lit- 
eracy skills and tested literacy skill levels (Kirsch, Jungeblut, Jenkins, and Kolstad, 1993). 
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The LILAA Persistence Study 
Figure 2.2 



Literacy Test Age/Grade Equivalents 
for English Speakers and English Learners 



□ English speakers ■ English learners 




PPVT Age Equivalent 







Grade I or Grade 2 Grade 3 Grade 4-5 Grade 6-7 Grade 8 or 

below higher 

ABLE Grade Equivalent 




Grade I or Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 or 

below higher 

TOWRE SWE Grade Equivalent 




below higher 

TOWRE PDE Grade Equivalent 

SOURCE: LILAA Achievement Study Database. 

NOTES: Sample sizes varied between the tests, from 192 to 241. 

Half the respondents were between ages 34 and 52; a quarter were older, and a 
quarter were younger. 
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The LILAA Persistence Study 
Figure 2.3 

Literacy Survey Responses: 

’’What do you find hard about reading?” 



I can't read/pronounce the long, hard words 
Spelling problems 
I read very slowly 
Sometimes I don't understand what I read 
I can't understand most of the hard words 
I forget a lot of what I read right afterwards 
I forget a lot of what I read a few days later 
Other problems 




0 10 20 30 40 50 60 70 80 90 



100 



Percentage of Respondents 



SOURCE: Survey of 199 LILAA achievement study participants. 
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The LILAA Persistence Study 
Figure 2.4 

Literacy Survey Responses: 

"What do you find easier now than before you entered this program?" 



I can understand more of what I read 
I can read/pronounce the long, hard words more now 
I can understand many of the hard words 
I read much faster 
I can spell better than before I started at the library 
I can remember more of what I read right afterwards 
I remember more of what I read a few days ago 
Other improvements 




10 20 30 40 50 60 70 80 90 100 



Percentage of Respondents 



SOURCE: Survey of 199 LILAA achievement study participants. 
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barriers and distractions that could interfere with their ability to persist. What were the existing 
patterns of persistence among participants as the LILAA programs began to evaluate students’ 
barriers to persistence and to implement interventions aimed at addressing those barriers? In a 
sense, what was the baseline level of participation? Do adult learners use library literacy pro- 
grams in different ways? Is there variation in participation for key subgroups of students served 
by the LILAA programs? 

As Chapter 1 discussed, available research suggests that there is a threshold of partici- 
pation that adult learners must meet before they are likely to make substantial gains in literacy. 
Defining persistence is not simple, but measures of students’ participation in terms of how long 
(the duration of their participation in the program) and how much (their total hours of activity) 
capture key aspects of this threshold.'® 

How Long Did Students Participate in the Library Programs? 

Students who entered a LILAA program were most likely to stop participating in the 
first three months; of all entering students who were active for one month, 77 percent partici- 
pated in the next month, and about two-thirds participated in the third month. Figure 2.5 shows 
how long students who entered the LILAA programs in the first year of this study participated 
in program activities. The horizontal axis identifies months relative to students’ entry into the 
programs (rather than calendar months)," while the vertical axis identifies the percentage of 
these students who were active for at least one hour in the various months of follow-up. 

After the first three months, average participation rates declined more slowly, partly be- 
cause students who made it through the first several months were more likely to remain engaged 
and partly because some who stopped participating in earlier months returned. In fact, as is dis- 
cussed later in this section, more than a third of entering students who dropped out at some 
point in the 18-month follow-up period used in this analysis subsequently returned to their pro- 
grams. However, despite these returns, the average participation rate continued to decline over 



'®These measures do not capture all aspects of student work. For example, they do not capture the 
persistence of students who leave a LILAA program to pursue further education elsewhere. Nor do they 
capture literacy activities done on students’ own time or any activities not tracked in the libraries’ data 
systems. The findings presented here are based on the program entrant short follow-up sample (n = 
2,1 10) and the program entrant extended follow-up sample (n =1,256), described in Appendix B. A dis- 
cussion of these definitions, other data issues, and the limitations of participation data can be found in 
Appendix C. Students are included in the entrant sample if they begin participation after a period of three 
months or more without program activity. Some who fit this definition have participated in the program 
in the past and, therefore, are more properly called “reentrants.” 

"Thus, Month 1 is the first month of participation in the program; Month 2 is the month following 
program entry; and so forth. Other figures in this chapter follow the same pattern. 
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The LILAA Persistence Study 
Figure 2.5 



Percentage Participating Over Time 
for Program Entrants in the Extended FoIIow-Up Sample 




SOURCE: MDRC calculations from site-reported participation data collected between January 2000 and 
December 2001. 

NOTES: The Extended Follow-Up Sample includes 1,256 new or returning students who had at least 1 8 months 
of data available. Some students may have had only 12-17 months of data available. (See Appendix B for a full 
description of the sample.) 

The month of follow-up is defined relative to the student’s first month of participation; thus, Month 1 
is the month that the student entered the program, Month 2 is the month following program entry, and so on. 
Month 1 could be a different calendar month for different students. 
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time. By the end of the first year of follow-up, less than 30 percent of the program entrants were 
still active, and, by Month 18, this number had shmnk to 15 percent. 

These initial findings highlight the importance of establishing a strong connection with 
learners during the first months after they enter a program. Some of these early departures may 
reflect the decisions of learners, who soon realize that these programs do not fit their needs and 
seek literacy instmction elsewhere. These are the short-term participants discussed later in this 
chapter. However, the findings also suggests that stemming the rapid decline in participation is 
an important aspect of improving persistence. Several of the enhancements discussed in Chapter 
3 and the early lessons posed in Chapter 4 aim to build a stronger initial connection with these 
learners. The high frequency of returning learners (about one-third of those who leave) also 
highlights the importance of methods to maintain contact with intermittent students (discussed 
later in this chapter) and to encourage them to “try again” when circumstances permit. Long- 
term participants are also represented in this sample, with 15 percent of entrants still attending 
some activities 18 months after they enrolled in the program. 

How Often Did Students Leave and Return to a Program? 

As shown in Table 2.2, nearly all program entrants (94.7 percent) left their program for 
at least one month within the 18-month follow-up period, and this percentage did not vary 
greatly across the sites. On average, entering students remained active 5.1 months before ini- 
tially dropping out.'^ About one-third (36 percent) of those who stopped participating for at least 
a month subsequently returned to the program at some point within the 18 months of follow-up. 
The longer students were away, the less likely they were to return. Only 20. 1 percent of students 
who were gone for two months or more ever returned; after six months, this percentage dropped 
to 6.2 percent. The average length of the gap between dropping out and then returning was 2.5 
months, but in more than half of all cases (54.6 percent), the time between leaving and returning 
was j ust one month (not shown in table) . ‘ ^ 



'^However, the median time of initial participation was four months (not shown in table), indicating 
that the average is somewhat skewed by some students who participated for very long periods before 
dropping out. 

'^It is important to realize that many of these one-month gaps may actually be the result of data prob- 
lems (for example, a tutor who failed to report hours of participation for a particular month). It is not 
possible to distinguish such data problems from real interruptions in participation. For more discussion on 
how these data issues affect the analyses in this chapter, see Appendix C. However, to get an idea of the 
impact that one month of missing data may have, the analysis in Table 2.2 was repeated, defining “drop- 
out” as a gap in program participation of two months or more. Under this definition, on average, most 
entering students (91 percent) still dropped out before the end of the 18-month follow-up period, but only 
20 percent of these students later returned, compared with 36 percent under the one-month definition of 
dropout. If students did not participate for two months, the chances of their returning were thus substan- 

(continued) 
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The LILAA Persistence Study 
Table 2.2 



Participation Patterns in the First 18 Months 
for Program Entrants in the Extended Follow-Up Sample 



Outcome 


Average 


Sample Size 


Average length of first spell of participation (months) 


5.1 


1,256 


Ever left program for 1 month or more (%) 


94.7 


1,256 


Of those who left, ever returned; 






After leaving for 1 month or more (%) 


35.6 


1,189 


After leaving for 2 months or more (%) 


20.1 


954 


After leaving for 6 months or more (%) 


6.2 


791 


For those who left and ever returned, average length 
of first break in participation (months) 


2.5 


423 



SOURCE: MDRC calculations from site-reported participation data collected between January 
2000 and December 2001. 

NOTES: The Extended Follow-Up Sample includes 1,256 new or returning students who had at 
least 18 months of data available. Some students may have had only 12-17 months of data 
available. (See Appendix B for a full description of the sample.) 



These early quantitative findings are consistent with the qualitative research findings 
presented later in this chapter. That analysis notes how some adult learners (both the short-term 
and the intermittent learners) move in and out of programs as their personal circumstances, in- 
terests, and needs change. Together, these early findings suggest that programs should have a 
strategy for maintaining contact with students who drop out and for welcoming them back when 
they do return. Further, because most “returnees” become active again after a relatively short 
gap (three months or less), library programs might want to devote special outreach efforts to 
absent learners after this initial period, in hopes of stimulating more returns after the period dur- 



tially lower. In addition, the average time that entering students remained active before dropping out for 
one month or more was 5.2 months; before dropping out for two months or more, entering students re- 
mained active for an average of 6.4 months. 
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ing which they typically occur. Making sure that learners know they are welcome to come back 
might be a way to raise the return rate. 

What Were the Initial Trends in Length of Participation? 

As the LILAA programs began to implement changes intended to boost persistence, 
there was no clear trend in how long students participated, and much of the variation that did 
exist in the early months appears to have been a result of varying lengths of participation for 
students entering programs at different times of the year. Controlling for this seasonal variation, 
there are some hints of an encouraging trend in the percentage of students participating at least 
one month following program entry, but at this point there are not similar encouraging findings 
at three and six months of follow-up. 

Figure 2.6 shows the rates of participation over time for six early cohorts of entrants. In 
this analysis, learners are grouped according to the calendar quarter when they entered. For ex- 
ample, these data show that 79 percent of the cohort of students who entered a LILAA program 
between January and March 2000 (the leftmost bar in each grouping of bars in the figure) were 
still active in the program in the month after entering; 56 percent were still active in the third 
month after entering; and 28 percent were active in the sixth month after entering. 

The quarter in which a person enters a program appears to have an effect on participa- 
tion. For example, those entering between April and June (the second bar in each grouping) 
were least likely to persist for one month or three months, while those entering between July 
and September (the third bar in each grouping) were most likely to persist. Some of these sea- 
sonal differences may be explained by students’ schedules at different times of the year.'"' How- 
ever, some of the variation across groups may be explained by other trends, such as changes in 
the economy, and by improvements in the completeness of the data over time as LILAA pro- 
grams refined their operations. These findings on seasonal variation are just suggestive at this 
point, but if they hold up with longer follow-up, they could imply that programs might need to 
devote special efforts to making a connection with entrants who arrive at particular times of the 
year and might need to focus their outreach efforts to attract new participants at times of the 
year when people are most likely to maintain a connection with the program.'^ 



'“'Because of the location of the programs in places with relatively mild climates (California, North 
Carolina) or good public transportation systems (New York City), the seasonal variation is less likely to 
be explained by weather differences. These seasonal differences in participation may be related to differ- 
ences in schedule demands in periods after participation starts. For example, parents who begin their lit- 
eracy studies in the summer may find it easier to continue in the fall as school starts for their children, 
whereas those who begin in the spring soon face summer school vacation, when the demands of child 
care are likely to increase. 

'^Other groupings of months besides calendar quarters were examined — including a division that 
isolated the summer months — with similar results. 
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The LILAA Persistence Study 
Figure 2.6 



Percentage Participating Over Time 
for Program Entrants in the Short Follow-Up Sample, 
by Quarter of Program Entry 



100 n 



90 - 




One Month After Entry Three Months After Entry Six Months After Entry 
(Month 2) (Month 4) (Month 7) 



Quarter of Program Entry 

EDJan-Mar'OO CDApr-Jun'OO DJul-Sep'OO BOct-Dec'OO HJan-Mar'Ol BApr-Jun'Ol 



SOURCE: MDRC calculations from site-reported participation data collected between January 2000 and 
December 2001. 

NOTES: The Short Follow-Up Sample includes 2,1 10 new or returning participants entering the programs 
between January 2000 and June 2001. (See Appendix B for a full description of the sample.) 
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Some of the variation in length of participation across cohorts may also be the result of 
changes in the programs, including but not limited to the implementation of interventions aimed 
at increasing student persistence. Because many of these interventions are still in the beginning 
stages of implementation, it is too early to try to link changes in persistence over time to the ef- 
fects of these interventions. However, the participation rates in the first month of follow-up do 
hint at an encouraging early trend: The first-month participation rates are higher for the Januaty- 
March 2001 cohort and for the April- June 2001 cohort than for their counterparts who entered 
the programs a year earlier. At this point, this suggestive finding does not hold for the participa- 
tion rates at three and six months after program entry — where a declining trend may be appear- 
ing. Analysis in the final report will include additional cohorts who entered at later points in 
time, when the interventions may have greater impact on program activities, and will assess 
whether stable trends emerged and seasonal effects persisted. 

How Much Time Did Literacy Students Spend in Literacy Activities? 

Intensity of effort is also an important factor in determining literacy improvements, and 
this analysis focuses on two measures of intensity: (1) total hours of participation across the en- 
tire follow-up period and (2) average hours of participation during months when a student is 
active. The first measure is an indicator of the overall level of students’ activity: How much 
time do they spend in program activities over an extended period? The second measure ac- 
knowledges that some students move in and out of active status and describes the intensity of 
their participation when they are active. 

Total Hours of Participation. Students in the LILAA persistence study averaged 57 
total hours of participation over 18 months, but influencing this average was a small number of 
students whose hours were quite high (the long-term participants), and a third of the sample 
who dropped out quickly (the short-term participants). Table 2.3 presents average total hours of 
participation and information about the distribution of total hours. Illustrating the substantial 
range of participation, 21 percent had over 80 hours of activity. At the other end of the distribu- 
tion, 24 percent of the students had 10 hours or less of participation during this 18-month fol- 
low-up period. The median student had only 3 1 hours of participation (not shown in table). 

Do library literacy students gradually participate less and less and then finally drop out, 
or do they abmptly leave a program? For these students in the LILAA programs, there was not a 
gradual tapering off of hours of participation prior to departure, and there is little evidence that 
the intensity of participation eroded before students left. Instead, students seem simply to have 
disappeared one day — a phenomenon that would severely limit the staffs ability to foresee 
and prevent such sudden departures. 
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The LILAA Persistence Study 
Table 2.3 



Total and Average Monthly Hours of Participation in the First 18 Months 
for Program Entrants in the Extended Follow-Up Sample 



Outcome 


Average total hours of participation 


57.1 


10 hours or less (%) 


24.3 


11-30 hours (%) 


24.8 


31-60 hours (%) 


21.3 


61-80 hours (%) 


9.0 


81 hours or more (%) 


20.6 


Average monthly hours of participation, when active 


7.1 


3 hours or less (%) 


23.5 


4-6 hours (%) 


35.0 


7-9 hours (%) 


19.2 


10-12 hours (%) 


10.0 


1 3 hours or more (%) 


12.3 


Sample size 


1,256 



SOURCE: MDRC calculations from site-reported participation data collected between January 
2000 and December 2001 . 



NOTES: The Extended Follow-Up Sample includes 1,256 new or returning students who had at 
least 18 months of data available. Some students may have had only 12-17 months of data 
available. (See Appendix B for a full description of the sample.) 

The average monthly hours of participation is calculated only for months in which the 
student was active. 



At best, programs might seek to learn about aspects of students’ lives that could pose a 
risk to continued participation (for example, fragile child care arrangements or potential health 
problems). However, these efforts might well be seen as inappropriately intrusive and might 
push learners away from the program, so there is no easy way to foresee departures. Regularly 
asking students in a general way about whether or not they are having trouble persisting might 
ensure that they at least have an opportunity to bring up any problems, if they choose to do so. 
Again, the best response may be to send a clear message that learners can and should remain in 
contact with the program even if they cannot participate actively. 
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Hours of Participation While Active. Shifting to measures of intensity of participation, 
Table 2.3 also shows average monthly hours of participation across months in which students 
were active. On average, students participated for approximately 7 hours in each month that they 
were active. This translates into less than 2 hours per week. In an average month, 24 percent of 
students participated for 3 hours or less, and 22 percent participated for 10 hours or more.'* 

Students who persisted longer also participated more intensely, right fi-om the start. For 
example, students who participated continuously for at least three months averaged more than 9 
hours of participation per month, and students who participated continuously for at least a year 
averaged about 12 hours per month (not shown in table). While students remained active, the 
intensity of participation did not consistently increase or decrease over time.” 

Thus, there is a positive relationship between hours of participation (even in the early 
months) and persistence. Students who started out participating more were more likely to stick 
with the program. It is not yet possible to determine, however, whether intensive participation 
actually increases persistence or whether intensive participators are simply different fi’om other 
learners at the outset. 

What Were the Initial Trends in Intensity of Participation? 

Were there differences in participation intensity among entrants who joined the LILAA 
programs as the programs gradually began to implement strategies designed to improve persis- 
tence? As with the earlier look at length of participation, there is no clear trend over time, but 
once seasonal variation is controlled for, there are some hints of an encouraging initial trend as 
programs began to address problems of persistence. 

Figure 2.7 shows the average monthly hours of participation one, three, and six months 
after program entry for groups of students entering LILAA programs at different times. As for 
other measures in this section, the focus is on hours of participation during months when a stu- 
dent was active in the program. This figure reinforces the finding that students who continued 
their participation over several months managed to maintain the intensity of their participation. 
For example, those students who entered a LILAA program between January and March 2000 
and remained active in the subsequent month (represented by the leftmost bar in each grouping 
of bars in the figure) participated for an average of 8 hours during that month. Students in this 



'*These statistics, however, include some people who may have started midway through Month 1, 
and they also include people who attended only once and never came back. 

'^Analyzed a different way, students who averaged fewer than 6 hours of activity per month over the 
first three months stayed in the program for an average of 4.4 months before leaving for the first time, 
while those who participated more than 8 hours per month in the first three months stayed in the program 
for an average of 6.9 months before their first break (not shown in table). 
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The LILAA Persistence Study 



Figure 2.7 

Average Monthly Hours of Participation Over Time 
for Program Entrants in the Short Follow-Up Sample, 
by Quarter of Program Entry 
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SOURCE: MDRC calculations from site-reported participation data collected between January 2000 and 
December 200 1 . 

NOTES: The Short Follow-Up Sample includes 2,1 10 new or returning participants entering the programs 
between January 2000 and June 2001. (See Appendix B for a full description of the sample.) 

The average monthly hours of participation is calculated only for sample members who were active 
in that month. 
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cohort of entrants who were active in their third month after entry (the middle set of bars) par- 
ticipated for an average of 9 hours per month; those in this cohort who were active in their sixth 
month after entry (the rightmost grouping) participated for an average of 7 hours per month. 

Figure 2.7 also illustrates that students who entered LILAA programs in the first and 
second quarters of 2001 were active for more hours than their counterparts who entered during 
the same quarters in 2000. (Because of the likelihood of seasonal variation, it is important to 
compare entrants fi'om the same period in each year.) In all three snapshots in time — one, 
three, and six months after entry — those who entered in 2001 participated for more hours per 
month than those who entered in 2000. This observation suggests that the interventions under- 
taken in the LILAA initiative might have begun to have an effect on the intensity of participa- 
tion among active students. Analysis of trends over time in terms of length and intensity of par- 
ticipation will be an important part of the final report. 

This suggestion of an early trend toward increases in the intensity of participation 
while students were active contrasts with the lack of an encouraging early trend in the per- 
centage of students who were active at three and six months of follow-up. The finding re- 
ported earlier in this section — that students seem to participate steadily and then abruptly 
disappear fi'om the program — could be a partial explanation. If students leave a program be- 
cause of difficult-to-anticipate changes in their lives (rather than because of a gradual decline 
in their ability to participate), then increasing the average length of participation could be 
much harder to do than raising the average intensity of participation while active in the pro- 
gram. When students are active, program features and support can help boost the amount of 
their participation. But if unexpected events outside the control of a program can end a stu- 
dent’s participation, it will be harder to extend the length of participation. 

Did Length of Participation Vary Across Subgroups of LiLAA 
Students? 

Did subgroups of LILAA students persist in their literacy activities at different rates? 
For example, did females tend to drop out earlier or to persist longer than males? For some sub- 
groups (such as those defined by gender), the answer is relatively simple, because the sample is 
spread fairly uniformly across the sites. As Figure 2.8 shows, males and females in the five 
LILAA programs remained active at similar rates.'® However, for other subgroups — such as 
those defined by ethnicity or primary mode of instmction — the local programs varied substan- 
tially; some served predominately one ethnic group or used one mode of instruction. Thus, in 



'®The differences between the two lines shown in Figure 2.8 are not statistically significant, with a 
few small exceptions during the earlier months. 
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The LILAA Persistence Study 
Figure 2.8 



Percentage Participating Over Time 
for Program Entrants in the Extended FoIIow-Up Sample, 

by Gender 




Month 



SOURCE: MDRC calculations from site-reported participation data collected between January 2000 and 
December 200 L 

NOTES: The Extended Follow-Up Sample includes 1,256 new or returning students who had at least 18 months 
of data available. Some students may have had only 12-17 months of data available. (See Appendix B for a full 
description of the sample.) 

The month of follow-up is defined relative to the student's first month of participation; thus. Month 1 
is the month that the student entered the program. Month 2 is the month following program entry, and so on. 
Month 1 could be a different calendar month for different students. 

Seven students were missing gender data and are not included in these calculations. 
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these cases, it can be hard to disentangle subgroup differences from local program differences. 
Despite these complexities, however, some patterns emerge. 

As shown in Figure 2.9, younger students tended to drop out at a faster rate than older 
students.'® For example, after six months, 50 percent of students between ages 36 and 50 were 
still participating in literacy activities, compared with only 38 percent of students under age 21 
and only 35 percent of students between ages 22 and 35. It is possible that younger students had 
alternative educational resources available to them, which could draw them away from these 
particular programs without really reflecting lower persistence in learning. This is illustrated by 
one of the case studies presented in the final section of this chapter, in which a younger student 
shifted to another program in the community in order to be with more students his own age. 

Employment status at program entry also seems to have created slightly different pat- 
terns of persistence, although perhaps not as one would expect. Figure 2.10 shows that students 
who reported that they were employed full time when they enrolled in library literacy programs 
dropped out at a slightly slower rate than those who reported part time or no employment.^ For 
example, six months after starting a program, 61 percent of students who were employed full 
time were still participating, compared with 57 percent of students who were not employed at 
the time of their enrollment and 55 percent of those who were employed part time at entry. Nine 
months after starting a program, 51 percent of students who were employed full time at entry 
were still participating, compared with 42 percent of students who were not employed and 44 
percent of those who had been employed part time at entry.^' 

One might expect that individuals who have jobs would find it more difficult to devote 
hours to a program because of the time conflicts they face. However, in other ways, students 
who are working when they enroll may be better equipped to persist in a literacy program. For 
example, they may be more accustomed to a routine schedule. Furthermore, their ability to hold 
a job may indicate that they do not face child care responsibilities or health problems or that 
they have support services or assistance from friends and family that help them take on demand- 
ing activities like work and program participation. 

For ethnic subgroups, local program differences in the ethnic composition of learners 
come into play in the analysis. As Figure 2.1 1 shows, there are differences in participation rates 
across the ethnic subgroups. These are statistically significant, even when controlling for differ- 
ences in the distribution of ethnic groups across the library programs (not shown in figure). 



'®After Month 2 of follow-up, the differences in participation rates across the age categories are sta- 
tistically significant at the 1 percent level. 

^®Note that Figure 2.10 shows participation only for those students for whom information on employ- 
ment was available at the time of program entry. This excludes 433 students across the nine programs. 

^‘This pattern continues through Month 14 and is statistically significant in Months 8, 9, and 12. 
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The LILAA Persistence Study 
Figure 2.9 



Percentage Participating Over Time 
for Program Entrants in the Extended FoIIow-Up Sample, 
by Age Group 




Month 

SOURCE: MDRC calculations from site-reported participation data collected between January 2000 and 
December 2001. 

NOTES: The Extended Follow-Up Sample includes 1 ,256 new or returning students who had at least 1 8 months 
of data available. Some students may have had only 12-1 7 months of data available. (See Appendix B for a full 
description of the sample.) 

The month of follow-up is defined relative to the student's first month of participation; thus, Month 1 
is the month that the student entered the program. Month 2 is the month following program entry, and so on. 
Month 1 could be a different calendar month for different students. 

Thirty-five students were missing age data and are not included in these calculations. 
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The LILAA Persistence Study 
Figure 2.10 

Percentage Participating Over Time 
for Program Entrants in the Extended Follow-Up Sample, 
by Employment Status 




SOURCE: MDRC calculations from site-reported participation data collected between January 2000 and 
December 200 1 . 

NOTES: The Extended Follow-Up Sample includes 1,256 new or returning students who had at least 18 
months of data available. Some students may have had only 12-17 months of data available. (See Appendix B 
for a full description of the sample.) 

The month of follow-up is defined relative to the student's first month of participation; thus, Month 1 
is the month that the student entered the program, Month 2 is the month following program entry, and so on. 
Month 1 could be a different calendar month for different students. 

A total of 433 students were missing employment data and are not included in these calculations. 
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The LILAA Persistence Study 
Figure 2.11 



Percentage Participating Over Time 
for Program Entrants in the Extended Follow-Up Sample, 

by Ethnicity 
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SOURCE: MDRC calculations from site-reported participation data collected between January 2000 and 
December 2001. 



NOTES: The Extended Follow-Up Sample includes 1 ,256 new or returning students who had at least 1 8 
months of data available. Some students may have had only 12-17 months of data available. (See Appendix 
B for a full description of the sample.) 

The month of follow-up is defined relative to the student's first month of participation; thus, Month 
1 is the month that the student entered the program. Month 2 is the month following program entry, and so 
on. Month 1 could be a different calendar month for different students. 

A total of 149 students were missing ethnicity data and are not included in these calculations. 




-28- 



47 



However, the differences are generally small and are not consistent.^^ At this stage, these find- 
ings do not suggest any particular programmatic response. 

Finally, there are clear differences in participation when entrants are disaggregated by 
their program’s primary mode of instmction: classes taught by professional instmctors, tutoring 
done by volunteers (either paired tutoring or in small groups), or computer lab (used independ- 
ently or in small groups).^^ Figure 2.12 shows that entrants whose primary mode of instmction 
was tutoring by volunteers were much more likely to participate over time than students whose 
primary mode of instmction was the computer lab. It also shows that those primarily working in 
computer labs were more likely to participate than entrants primarily participating in classes 
taught by professional instmctors.^'* For example, by Month 2, only 52 percent of students partici- 
pating primarily in classes were still active in the program, whereas 85 percent of students partici- 
pating primarily in tutoring stayed active. These large differences remained over time: 20 percent 
in classes versus 57 percent in tutoring were active in Month 6, and 1 1 percent versus 43 percent, 
respectively, were active in Month 12.^^ Throughout the 18-month follow-up period, the participa- 
tion rate of students who were primarily in computer labs fell between these two extremes. 

These findings should be seen as preliminary for several reasons. It is hard to separate the 
effects of the primary mode of instmction fix)m differences across local programs; though tutoring 
is the central element of most programs, the sites vary in their use of supplemental methods such 
as classes and computer labs. Further, students can participate in different ways in these modes of 
instmction; computers may be used by students awaiting a tutor or by more advanced students, for 
writing and preparing for the GED or other tests. Likewise, classes taught by instmctors can be 
supplementary, focusing on specific skills such as spelling or on specific content areas, or students 
may attend classes as they await the assignment of a tutor. The final report will present further 
analysis of the participation data, by predominate mode of instmction, as well as field research on 
the ways in which different programs used the various instmctional modes. 



^^The one exception is for white students, who make up such a small percentage of all entering stu- 
dents (approximately 5 percent) that it is difficult to interpret these differences. 

^^The primary mode of instmction is defined as the one in which students spent the most hours dur- 
ing the 18-month follow-up period. Computer lab usually entails computer-assisted instmction, though 
staff in one site guide the students’ use of the computer, and hours spent writing on the computer are in- 
cluded in this category. 

^*These differences are statistically significant. 

^^Note that it was more difficult for programs to capture time spent in tutoring and computer lab, but 
despite any underreporting relative to classes, these modes of instmction still led to greater persistence. 
For discussion of issues related to data limitations, see Appendix C. 
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The LILAA Persistence Study 
Figure 2.12 

Percentage Participating in Any Activity Over Time 
for Program Entrants in the Extended Follow-Up Sample, 
by Primary Activity 




Month 

SOURCE: MDRC calculations from site-reported participation data collected between January 2000 and 
December 2001. 

NOTES: The Extended Follow-Up Sample includes 1,256 new or returning students who had at least 18 
months of data available. Some students may have had only 12-17 months of data available. (See Appendix 
B for a frill description of the sample.) 

The month of follow-up is defined relative to the student's first month of participation; thus, Month 
1 is the month that the student entered the program. Month 2 is the month following program entry, and so 
on. Month 1 could be a different calendar month for different students. 

"Primary activity" is defined as the activity in which the student spent the most hours during the 18 
months of follow-up. 

A total of 135 students were missing primary activity data and are not included in these 
calculations. 
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Did Intensity of Participation Vary by Subgroup? 

Table 2.4 presents measures of total hours and average monthly hours of participation 
in active months for various subgroups of students; statistically significant differences across 
subgroups are indicated by stars. Key findings are the following: 

• Gender. Participation intensity for males and females was similar, both for 
total hours and average monthly hours while active. 

e Age. Older students participated for more total hours than younger students 
because, as discussed earlier, their participation was sustained over a longer 
time. However, average hours during active months did not vary in a statisti- 
cally significant way. 

• Ethnicity. There were statistically significant differences between ethnic 
subgroups in terms of both total hours and average monthly hours of partici- 
pation while active, even when controlling for the local program. Black stu- 
dents participated for the most total hours and had the highest average 
monthly hours, and Asian and Pacific Islander and white students had the 
lowest monthly hours of participation. As noted earlier, these differences 
could, in part, result firom program-specific characteristics, because the ethnic 
groups tend to be clustered by program and because the statistical controls 
for local programs do not completely isolate the effects of ethnic differences. 

• Employment Status at Program Entry. Total hours of participation did not 
vary significantly by initial employment status. Although employed students 
were shown to remain active longer, this did not translate into higher hours of 
participation overall, because nonemployed students had higher average 
hours during active months. 

It is interesting to note that few students who were not employed at the time of enroll- 
ment even approached “half-time” participation in these programs. Only about 9 percent of all 
participants who were not employed when they enrolled participated for more than 1 5 hours a 
month on average (not shown in table). This may partly reflect limitations of the databases 
underlying these figures, but it also reflects the operations of the library literacy programs in this 
study, which tend to rely on volunteer tutors who have limited hours to spend with students. As 
a result, even students who had relatively few other demands on their available time may not 
have participated iii activities for many hours a month. 

Table 2.4 also shows measures of average total monthly hours for students according to 
primary mode of instruction. The table includes hours for all student activities, not just hours in 
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The LILAA Persistence Study 
Table 2.4 



Total and Average Monthly Hours of Participation in the First 18 Months 
for Program Entrants in the Extended Follow-Up Sample, 

by Subgroup 





Total Hours 


Average Monthly Hours, 
When Active 




Subgroup 


Mean 


Median 


Mean 


Median 


Sample Size 


Gender 


Male 


57.0 


28.6 


7.3 


5.9 


495 


Female 


57.3 


33.4 


6.9 


5.7 


754 


Age group 


Under 2 1 


49.2 


31.6 


7.2 


6.0 


105 


21-35 


44.8 


23.0 


6.6 


5.5 


475 


36-50 


63.5 


>"*** 40.8 


7.2 


5.8 


440 


51 -65 


76.8 


53.2 


7.5 


6.1 


160 


Over 65 


82.8. 


45.8 


8.2 


6.9 


41 


Ethnicity 


Black 


74.5 


42.7 


8.4" 


7.0 


429 


Hispanic 


62.8 


42.9 


7.0 


5.8 


258 


Asian/Pacific Islander 


45.2 


>"*** 26.5 


5.8 


4.6 


322 


White 


49.0 


39.1 


5.6 


5.2 


48 


Other 


53.1, 


36.0 


7.1. 


5.9 


50 


Employment status 


Employed 


68.2 


47.7 


7.0 


5.9 


369 


Full time 


70.4 


47.5 


7.2] 


1 


236 


Part time 


64.4 


51.1 


6.8 


y** 6.1 


133 


Not employed 


73.4 


50.3 


8.1 J 


1 6.2 


454 


Primary activity 


Tutoring 


88.6] 


55.5 


8.6 1 


1 


347 


Classes 


29.3 


12.5 


6.3 


y*** 5.5 


169 


Computer lab 


49.3 J 


27.9 


6.7 J 


1 5.5 


, 605 


Full sample 


57.1 


31.3 


7.1 


5.8 


1,256 



SOURCE: MDRC calculations from site-reported participation data collected between January 2000 and 
December 2001. 



NOTES: The Extended Follow-Up Sample includes 1,256 new or returning students who had at least 18 
months of data available. Some students may have had only 12-17 months of data available. (See 
Appendix B for a full description of the sample.) 

The average monthly hours of participation is calculated only for months in which the student 
was active. 

Subgroup differences were evaluated using an analysis of variance. Statistical significance levels 
are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 
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the primary mode of instmction. It shows that students whose primary mode of instmction was 
tutoring participated for 8.6 hours per month, on average — 37 percent more time than students 
whose primary mode of instmction was classes and 28 percent more time than students whose 
primary mode of instmction was computer lab. These findings are statistically significant when 
controlling for program, and they further demonstrate that students’ primary mode of instmction 
may affect their persistence, not just in continuing to participate from one month to the next but 
also for participating more hours each month. Paired and small-group tutoring are associated 
with the highest monthly hours of participation. As noted earlier, differences in participation 
that are associated with modes of instmction will be explored further in the final report. 

Were There Initial Differences in Participation Across the LILAA 
Programs? 

So far, this chapter has presented findings on the retention of students and the intensity 
of participation for the sample of entering students who were active in any of the nine LILAA 
programs. The following sections look at some of the variation in these measures across the pro- 
grams. In the final report, it may be possible to link trends in participation at the program level 
to the interventions that the programs are implementing to increase student persistence. 

Length of Participation. As presented above, across all libraries, 77.2 percent of enter- 
ing students continued to participate in literacy activities past their initial month of activity. As 
shown in Figure 2.13, across the LILAA programs, 45 percent to 100 percent of students re- 
mained active in the second month.^® This variation may reflect the different subgroups served, 
but it may also reflect some variation in the effects that the programs had on students’ ability or 
desire to persist, at least at the start of their involvement. For example, programs that actively 
enhanced their orientation for entering students may have had more success in retaining stu- 
dents in the first months after enrollment. 

Figure 2.13 also shows the percentage of students who were active in Month 6, when 
participation ranged from 14 percent to 83 percent. In most cases, those programs with high 
rates for one-month retention also had relatively high rates for six-month retention and vice 
versa. But there are a couple of exceptions. For example, one program retained only 55 percent 
of entering students through Month 2, but it still retained 39 percent through Month 6. Again, 
this may reflect data issues and underlying differences in the characteristics of the students 
served, but it also may show that some programs may excel in getting students to stay active 



^®However, because two of the programs initially reported data for periods longer than one month, 
reported participation hours were spread evenly across the early months in the reporting period. There- 
fore, some of these very high retention rates are inflated. This occurred for new students at one library in 
2000 and at another library until May 2001. 
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The LILAA Persistence Study 



Figure 2.13 



Percentage Participating Over Time 
for Program Entrants in the Extended Follow-Up Sample, 
by Program 




SOURCE: MDRC calculations from site-reported participation data collected between January 2000 and 
December 200 1 . 

NOTES: The Extended Follow-Up Sample includes 1,256 new or returning students who had at least 18 months 
of data available. Some students may have had only 12-17 months of data available. (See Appendix B for a full 
description of the sample.) 

Sites sometimes reported participation data for a period longer than a single month. In these cases, 
hours were divided among the months. Thus, if a site reported in three-month increments, a student who was 
active at any point during that period would be considered active for all months in the period. This leads to some 
inflation of the percentage participating in the short term and helps explain why some sites appear to retain 100 
percent of their students into the second month of participation. 
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when they are new to literacy activities, while other programs may be more successful at keep- 
ing students over a longer period of time. This may be due to the types of activities that the pro- 
grams offer or to the other types of services in place, perhaps aimed at helping students to over- 
come short- or long-term obstacles to persistence. These possibilities will be explored in the 
final report. 

Interruptions and Returns. As previously discussed, 35.5 percent of entering students 
who dropped out of a literacy program for at least one month during the follow-up period also 
returned during the follow-up period. As shown in Table 2.5, however, behind these averages, 
there was substantial variation across the programs. For example, in Programs B and G (Figure 
2.13), most of the students continued to participate after one month, and a relatively large pro- 
portion of them (41 percent in both cases) returned to the program after an eventual intemiption 
in activity. In Programs H and I, by contrast, although most of the students also continued to 
participate after their first month, substantially smaller proportions (22 percent and 8 percent, 
respectively) returned after stopping participation. Program E lost a larger percentage of stu- 
dents after one month, but half the students who stopped participating at some point returned to 
the program. 

These different patterns suggest that student persistence can mean different things for dif- 
ferent programs and, therefore, that programs might pursue different strategies to increase persis- 
tence. In Programs H and I, students who persisted tended to participate month after month, main- 
taining fi-equent activity with the literacy programs; but once students left, they were unlikely to 
return. At other programs, like Program E, students who persisted may have stopped and started 
their participation, but they maintained it over a longer period, despite those intemiptions. These 
observations suggest that some programs might increase persistence by focusing on helping stu- 
dents to remain engaged, even if their circumstances make them miss a month, whereas other pro- 
grams might focus on preventing fi’cquent intemiptions in students’ participation. 

Intensity of Participation. As shown in Table 2.6, the total hours that students spent in 
literacy activities during the 18-month follow-up period also varied across the programs. Average 
total hours ranged fiom 22 to 98, but some programs had a substantial number of students partici- 
pating for many more hours. For example, 25 percent of entering students in Program B partici- 
pated for more than a total of 127 hours (not shown in table), which represents this program’s high 
retention rate and relatively high program intensity. Those programs that started with a relatively 
high intensity of participation, as reflected in average hours in the second month, were not all able 
to sustain this intensity over time, suggesting that efforts to support high intensity may need to 
take different forms as participation continues. These observed differences in intensity could indi- 
cate different program capacities to help students persist, in terms of dedicating substantial 
amounts of time to literacy activities, but they could also indicate different foci of the programs, in 
terms of the types of students served and the types and fi’equency of instmction offered. 



The LILAA Persistence Study 
Table 2.5 



Participation Patterns in the First 18 Months 
for Program Entrants in the Extended Follow-Up Sample, 

by Program 











Program 










Outcome 


A 


B 


c 


D 


E 


F 


G 


H 


I 


Average length of first 


spell of participation (months) 


2.9 


8.5 


5.6 


2.0 


3.2 


4.9 


7.5 


6.1 


6.7 


Ever left program for 1 month or more (%) 


98 


90 


95 


99 


96 


87 


97 


94 


84 


Of those who left, ever returned: 


After leaving for 1 month or more (%) 


34 


41 


35 


27 


53 


62 


41 


22 


8 


After leaving for 2 months or more (%) 


22 


7 


10 


19 


39 


48 


39 


19 


4 


After leaving for 6 months or more (%) 


5 


4 


3 


7 


10 


25 


20 


3 


4 


For those who left and ever returned, average 


length of first break in participation (months) 


2.4 


1.4 


1.7 


3.6 


2.6 


3.3 


4.0 


2.8 


4.5 


Sample size 


184 


217 


241 


178 


124 


52 


72 


157 


31 



SOURCE: MDRC calculations from site-reported participation data collected between January 2000 and 
December 2001. 

NOTES: The Extended Follow-Up Sample includes 1 ,256 new or returning students who had at least 18 months 
of data available. Some students from programs D, E, and I may have had only 12-17 months of data available. 
(See Appendix B for a full description of the sample.) 
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The LILAA Persistence Study 
Table 2.6 

Total and Average Monthly Hours of Participation in the First 18 Months 
for Program Entrants in the Extended Follow-Up Sample, 

by Program 





Total Hours 


Average Monthly Hours, 
When Active 


Hours in First 
Full Month 




Subgroup 


Mean 


Median 


Mean 


Median 


Mean 




Sample Size 


Program 

A 


35.3'' 


14.8 


6.6^ 


5.5 


9.0 




184 


B 


98.3 


59.3 


9.1 


6.9 


9.0 




217 


C 


45.4 


26.6 


4.8 


4.1 


4.5 




241 


D 


21.5 


10.0 


6.2 


4.9 


8.7 




178 


E 


64.0 


26.9 


7.4 


V"* 5.5 


12.4 




124 


F 


71.7 


41.5 


7.2 


6.0 


9.6 




52 


G 


74.0 


55.3 


8.0 


7.2 


7.4 




72 


H 


61.8 


28.5 


7.6 


6.6 


8.0 




157 


I 


76.5 j 


42.7 


10.4 J 


10.1 


10.4 J 




31 


Full sample 


57.1 


31.3 


7.1 


5.8 


8.1 




1,256 



SOURCE: MDRC calculations from site-reported participation data collected between January 2000 and 
December 2001. 

NOTES: The Extended Follow-Up Sample includes 1,256 new or returning students who had at least 18 
months of data available. Some students from programs D, E, and I may have had only 12-17 months of 
data available. (See Appendix B for a full description of the sample.) 

The average monthly hours of participation is calculated only for months in which the student was 

active. 

Subgroup differences were evaluated using an analysis of variance. Statistical significance levels 
are indicated as: *** = 1 percent; *♦ = 5 percent; * = 10 percent. 
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How DSd the Broader Context of Participants’ Lives Affect Their 
Persistence? 



The analyses presented above show that persistence in the LILAA programs is limited 
but that there is considerable diversity among students. Within this diversity, types of students 
emerge, and examples of key barriers to participation and supports appear.^’ These findings can 
help program staff, policymakers, and researchers develop more realistic strategies for support- 
ing students’ persistence and goals. 

From the perspective of program staff, students appear, disappear, and suddenly reap- 
pear many times, and staff may feel that they struggle to help students achieve higher literacy 
skills even though students show little consistency in attendance. Staff may not notice different 
participation patterns among students, and they may have to fight the tendency to label students 
who do not meet participation expectations as being somehow deficient. Little is known about 
students outside the context of the individual program — unless programs conduct brief inven- 
tories of goals, or if students are mandated to attend (and thus more is known about them 
through their social service link). 

Linking student persistence and participation to students’ lives outside the program al- 
lows for a multifaceted view of persistence that acknowledges differences arnong students. The 
brief case studies of students presented in this section reveal the ties between persistence in a 
program and the complexity of students’ outside lives, why and how students use literacy ser- 
vices to reach their goals, the barriers they face, and the resources they draw on as they seek to 
improve literacy skills. More important, the case studies help develop and illustrate categories 
of students who have similar needs and, therefore, can be served with common interventions. 

Patterns of Persistence: Case Studies 

At this stage in the project, the research team has identified four patterns of student per- 
sistence: intermittent, short term, long term, and mandatory. 

Intermittent Persistence. Intermittent persistence is illustrated by students who have 
joined a library literacy program, left it, and returned at a later time or several times. Intermittent 
students may maintain some form of connection to the program while they are not participating. 
They often leave because of a crisis or difficulty in their lives and then return when the problem 



^^The research team developed case studies of 30 students during the first year of the project (spring 
2000 to spring 2001), but this chapter focuses on only four of them. Additional findings will be presented 
in the final report for this project, to be completed in 2003. Of the participants in the 30 case studies, 11 
are African-Americans, 17 are immigrants (Asian, Afro-Caribbean, African, and Hispanic), and 2 are 
native-born Caucasians. 
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has passed, or they may leave temporarily or stay partly connected to the program if their tutor 
leaves and they are awaiting a new one. Intermittent students may be focused on either broad 
goals (such as improving their language and basic skills) or specific goals (passing a citizenship 
test). This group might persist longer if the program could maintain some form of contact with 
them, find out why they stopped, provide them with a way to continue learning during their 
time of nonparticipation, and welcome them back as soon as they are ready to return. If a stu- 
dent has left the program because his or her tutor departed, a quick rematch would be important. 
Support strategies might also be critical for intermittent students. 

Elena: FoUowing an Immigrant’s Dreams. Elena has been an intermittent 
participant at Project READ for two years. She moved to California fi’om El 
Salvador when she was 17 years old. At the time, she did not speak English 
and did not have a high school diploma. Now 26, Elena is working toward a 
number of specific literacy goals that will enable her to improve her career 
prospects, raise her income, and better support her children. Already she has 
passed the GED exam in Spanish, improved her English, and learned to read 
stories to her sons; currently, she is preparing for a pharmacy technician exam. 

Elena’s accomplishments have not come easily, and she works very hard. 
Balancing work with child care and her studies has been an enormous bur- 
den. Transportation has also been problematic. She has often had to miss tu- 
toring sessions, and sometimes she has just been too tired. Moreover, Elena 
has at times had to flee an abusive husband, who often prevented her from 
leaving the house and attending adult schools in the community. She has 
been in and out of shelters and has struggled to support her children. 

Yet Elena’s determination has led her to stick to her goals and continue her 
learning when she can. Even being put on the long waiting list for tutors at 
Project READ did not dissuade her. She was on the waiting list for two years 
before being assigned her first tutor, and when that tutor left the program, she 
was put back on the waiting list for another six months. During these periods, 
she found classes at other programs or participated in the library’s Story 
Hour program with her children. 

Despite a number of setbacks, Elena has been able to piece together a pattern 
of intermittent participation in literacy activities that has led to success. 

Along the way, she has had a lot of support fi’om her fiiends and her tutor. 

Three times a week, she visits her tutor’s home, where her youngest son 
watches TV while she studies. Her social workers and fiiends helped her 
leave her abusive husband and encourage her in her literacy activities. Some- 
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day, Elena hopes to help other women like herself by becoming a lawyer and 
defending victims of domestic abuse. 

Short-Term Persistence. Short-term persistence is demonstrated by students who enroll 
in a library literacy program and participate intensively for a short period of time in order to ac- 
complish a specific goal or to start a transition in their lives. That goal may be enrollment in an- 
other program, such as a GED preparation class, or the attainment of adequate skills for a job. 
Short-term students sometimes also engage in self-study. This group should be helped to move on 
if a different program is more appropriate for their needs. If these students attain their short-term 
goals or make a connection to a more appropriate program, their short period of participation 
should be viewed as a success. For this group, educational and vocational counseling as well as 
exit interviews would be important in learning about and documenting their program transitions. 

Larry: Finding a Stepping-Stone Toward Graduation. Larry, a 19-year- 
old recent immigrant fi-om Jamaica, attended literacy activities at the Adult 
Learning Center (ALC) of the Central branch of Queens Borough Public Li- 
brary. The ALC was just a 10-minute cab ride fi’om his home in Jamaica, 
Queens, and his mother funded the trips. Larry felt comfortable with his as- 
signed small tutoring group, and he enjoyed the Books-on-Tape and reading 
materials on Michael Jordan and Pele. However, because Larry had not fin- 
ished high school, he wanted a setting that was more like a school — some- 
place that he could “graduate” fi’om and meet others like himself There were 
few people his age at the ALC. 

Larry’s main goal has been to earn a GED. He considers himself, in his 
words, a “slow learner,” as he has always stmggled in school and with read- 
ing. But support fi’om family and fiiends instilled a strong determination to 
persist in his education. Although he did not remain in the literacy program 
offered by the Central branch of Queens Borough Public Library, the pro- 
gram served as a stepping-stone toward his goals and motivated him to find a 
program that better fits his needs. 

Long-Term Persistence. Long-term persistence is seen in students who are active in 
a library literacy program over a long period of time and who participate regularly in classes, 
tutoring, or computer-assisted instruction or other self-study activities. Long-term students 
may simultaneously use another program in the community as well. They usually do not have 
specific goals but, rather, are seeking a schooling experience or are pursuing larger life goals. 
In addition, some of these students may be using the program — and attending consistently 
— as a way to engage in human interaction, and programs should acknowledge that these stu- 
dents are receiving a valuable service that may not produce learning gains. Programs should 
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help long-term students to develop realistic, specific goals and to monitor their progress as 
well as offer them diverse learning opportunities. 

Opal: Finding a Community for Learning. Opal, an Afiican-American 
woman originally fi-om rural North Carolina, has lived most of her 80 years 
in the Bronx. After retiring in her early seventies fi'om work as a house- 
cleaner and cook. Opal decided to go back to school. She spent one year at- 
tending Bronx Community College, but when the program closed, she started 
classes and tutoring sessions at New York Public Library’s Center for Read- 
ing and Writing (CRW) at the Francis Martin branch (not part of the LILAA 
study) and immediately loved the staff and tutors. After a few years — de- 
spite a devastating setback when her literacy tutor was killed in a car accident 
— and the temporary closure of the Francis Martin branch’s center. Opal 
continued her literacy studies by transferring to the CRW at the library’s 
Fordham branch. 

In the three years that Opal has been at the Fordham CRW, she has become 
very close with her new tutor. She says that they share an interest in religion, 
a very important part of Opal’s life. Opal is also learning to read a variety of 
materials; she reads the Bible at home in the morning, because “the minute 
you drink that water, you OK — right the minute I read this. I’m OK.” She 
reads Time magazine too, because a family member works for the company. 

At the CRW, she reads books about the lives of Afiican-Americans who, like 
her, migrated fi-om the South and struggled to be successful — including an 
adult literacy version of a biography of Oprah Winfi-ey. Opal finds these texts 
challenging, and although she feels “slow,” she commented, “If I sit down 
and take my time, I can read it.” 

Opal would also like to learn more about computers but wishes that she had 
more individualized assistance because she doesn’t know how to operate 
them. She also feels finstrated with her math group and stopped attending it 
because it was back-to-back with another one and was overwhelming. Her 
lack of confidence in these two areas is her main barrier to learning, and she 
feels as though she needs more help. However, she continues to attend activi- 
ties at the Fordham CRW because she feels that it lets her experience the 
education she was denied as a child. Although she often becomes tired or 
gets headaches and doesn’t stay long, she doesn’t have many responsibilities 
that interfere with her being able to attend often. Although Opal says that she 
doesn’t have a specific goal, she is motivated to continue her studies by her 
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desire to read the Bible, by her role models, and by the staff. She plans to at- 
tend for as long as she can. 

Mandatory Persistence. Mandatory persistence is exemplified by students who must at- 
tend a literacy program because they are required to do so by a public assistance program. Their 
attendance is usually steady and long term, and their stated goals are often related to those of the 
assistance program. Library literacy programs should help these students form their own goals for 
participation, if they have not already established them, so that they can come to see mandatory 
participation as an opportunity rather than a punishment. Some of these students who would not 
normally enter a literacy program may get “hooked” and may start participating regularly, increas- 
ing their motivation to learn. In addition, once their public assistance ends and they become more 
independent, they may need additional assistance to persist in literacy studies. 

Mel: Moving from Mandatory to Motivated Learning. When Mel told his 
CalWORKs counselor that he could not read or write well enough to take the 
food service job that was recommended for him, the counselor told him to 
call Second Start. Hesitant, but without a choice, Mel sought help fi:om the 
literacy program. At 52, it was the first time that he had told anyone about his 
literacy problems. Not even his girlfiiend of four years knew about his read- 
ing difficulties. Until he found Second Start, Mel felt that he had been able to 
compensate for his lack of literacy skills with “street knowledge.” But now 
he appreciates a point that his counselor made, and he says: “It takes a man to 
realize that he needs help and to ask somebody for help.” 

Mel worked on the computers and was assigned a tutor three weeks after he 
started. He set a goal to attend every day that he could, four days a week. 

When his tutoring began, he was concerned about other people’s seeing him at 
a literacy program, preferring to keep to himself and to meet with his tutor in a 
private room. However, as he came to realize that the other students at Second 
Start were in the same situation — improving their literacy and working on 
their own problems — Mel soon came to feel comfortable. In addition to 
working with his tutor, Mel spends a lot of time in the computer lab, and he 
reads at home every morning. His goal is to be able to read the Bible. 

Mel feels that he has a number of “brothers,” or positive role models support- 
ing his efforts to learn to read, including his tutor and caseworker. They are 
meaningful to his persistence. He says that someday he would like to give 
back, as a counselor helping other students with whom he can identify. Al- 
though he attends Second Start because he is required to do so, he has come to 
terms with his learning needs and has found the personal motivation to persist. 



O 

ERIC 



- 42 - 



61 



Is 



Different Patterns, Different Approaches 

The foregoing case studies demonstrate that students arrive at a library literacy program 
with different histories, goals, levels of determination, and life responsibilities — all of which 
are important factors in persistence and may result in different patterns of persistence. From a 
programmatic point of view, a student with a short-term or an intermittent pattern of participa- 
tion may look like a failure, compared with a long-term participant who consistently attends one 
program. However, the short-term student may move on to another educational program, and 
the intermittent student may be making steady progress toward specific goals over a longer pe- 
riod of time. Conversely, some long-term students are seeking human interaction and personal 
growth — goals that are harder to measure. These case studies demonstrate that programmers 
may need to take different approaches to improve the persistence of students who show differ- 
ent patterns of participation. The case studies point out that success in improving persistence 
may take various forms. Despite the complexity of adult learners’ lives, the goal of this research 
is to establish a small number of patterns — though probably more than the four identified in 
the initial implementation phase of this study — that library literacy programs can use to match 
their services to the needs of a variety of students. 



Conclusion 

The picture presented in this chapter underscores the importance of both the LILAA 
persistence study and the Wallace-Reader’s Digest Funds initiative that underlies it. The data 
show considerable diversity — in the types of students who enroll in library literacy programs, 
in the degrees of their persistence, and in the patterns of their persistence. The data also show 
that, overall, persistence is low, for numerous and complex reasons. During the first 18 months 
of follow-up, an average student participated for about 5 months before leaving for the first 
time, and over one-third of those who left returned to participate in later months. In the months 
that they attended, students averaged approximately 7 hours of participation per month. These 
averages are skewed, so a typical (median) student would have somewhat shorter participation 
spells and fewer hours. Despite the diversity of participation, the overall levels of participation 
at the start of the LILAA project highlight the need to increase student persistence. Also, the 
various patterns of persistence found among the case studies highlight the need to differentiate 
approaches in increasing persistence. Given that most students start out reading at the third- or 
fourth-grade level, they most likely need substantially more hours of literacy instruction than 
they currently receive in order to make significant progress in their literacy skills. 
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Chapter 3 

What Are the LILAA Programs Doing 
to Improve Student Persistence? 



As highlighted in Chapter 2, the adult students in the Literacy in Libraries Across 
America (LILAA) programs have very low literacy levels. Typically, they score at level 1 or 
level 2 of the National Adult Literacy Survey (NALS).‘ Generally, it would be very difficult for 
students who read at these levels to improve their literacy and achieve their goals if they partici- 
pate in a literacy program only sporadically or for short periods. The findings in Chapter 2 about 
the diversity of the students’ backgrounds, the types of barriers they face, and their patterns of 
persistence demonstrate the importance of addressing persistence in library literacy programs 
and of finding approaches that fit a variety of needs. Through the LILAA initiative, the Wal- 
lace-Reader’s Digest Funds is helping the library literacy programs in this study address the 
pressing issue of persistence among adult students. 

At the center of the initiative are the tasks of designing and implementing specific 
strategies for improving student persistence. These strategies are aimed either at addressing a 
range of barriers that impede students’ participation in a library literacy program or at enhancing 
supports that bolster participation. After just 18 months of planning and implementation, it is 
too early to assess the effects that any of these strategies have had on students; however, it is 
already evident that participation in this study, with its focus on persistence, has strengthened 
these nine well-established library literacy programs. In devising strategies, the programs in this 
study have gained a deeper understanding of the barriers and supports to continued participation 
and have sharpened their focus on the importance of persistence. At the same time, the libraries 
have also discovered the complex challenges in implementing new approaches and finding what 
best fits the needs of their students and program. This chapter analyzes the LILAA programs’ 
experiences and challenges in the initial implementation phase of the persistence study, fi'om 
January 2000 through June 2001. 



What Kinds of Strategies Do the LILAA Programs Use? 

During the design phase of the project, the Wallace-Reader’s Digest Funds and the 
LILAA program directors agreed on a set of four categories to guide them in selecting and 

*In 1992, the U.S. Department of Education undertook the National Adult Literacy Survey, which 
measured literacy and math skills using a fivedevel scale. Each NALS level represents a range of skills. 
According to the survey, 21 percent to 23 percent of the U.S. adult population would score in level 1, and 
another 25 percent to 28 percent would score in level 2. See Kirsch, Jungeblut, Jenkins, and Kolstad, 1993. 
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planning strategies for the initiative. Each program was expected to tailor strategies to address 
the unique needs of its learners, community, and institutional context; however, defining com- 
mon categories encouraged the programs to focus on specific issues associated with the persis- 
tence of adult students and created learning opportunities for the field by establishing compara- 
ble strategies across the sites. The four categories of strategies used by literacy programs in the 
LILAA persistence study are defined below. 

® Information-Gathering Strategies.^ These refer to efforts to gain more 
knowledge about student persistence. Included are strategies for improving 
communication with other programs in order to expand learning opportunities. 

For this initiative, many strategies are intended to help die programs gather in- 
formation that can guide them in determining what other strategies are needed 
to increase persistence. Some information-gathering strategies also allow pro- 
grams to monitor the progress of their students and to assess improvements in 
persistence so that they can continually refine their approaches. 

e Instructional Strategies.^ These are aimed at enhancing programs’ core in- 
structional services in order to better engage students and encourage contin- 
ued and fi-equent participation. Instructional strategies aim to find the best fit 
between learners’ needs and learning styles and the program’s offerings so 
that students will be more engaged in learning and will find it more reward- 
ing to persist longer. 

e Operational Strategies. These involve changes in program procedures to 
make activities more accessible to students and, thus, to make attendance 
more feasible. These types of strategies — such as extended hours or days of 
operation or additional locations — attempt to adjust the program’s offerings 
to the lives and schedules of the students. 

e Support Strategies. These are aimed at alleviating barriers to learner persis- 
tence, by providing either new or enhanced services. Some support strategies 
may address barriers caused by competing demands on students’ lives; for 
example, students may lack child care or may not have adequate transporta- 
tion. Other support strategies are less tangible and help reduce psychological 
obstacles to persistence, perhaps related to students’ prior failures in formal 
education that undermined their self-confidence. 



^In prior LILAA reports and documents, these are called “informational strategies,’’ but this new ter- 
minology more accurately conveys what these approaches involve. 

^In prior LILAA reports and documents, these strategies are called “programmatic strategies,” but 
this new terminology more accurately conveys the nature of these efforts. 
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The original concept for the project was that each program would seek advice and in- 
formation fix)m a variety of sources — including students and tutors — and would use what was 
learned to design strategies in each category; then the program would implement the strategies 
over a two-year period. The outcomes would be assessed by looking at student record data to 
analyze changes in participation and by observing changes in participants’ experiences. The rest 
of this chapter presents not only the strategies that were originally proposed by the programs 
and the strategies that were later added but also a number of approaches that were already in 
place and that tend to be innate to adult library literacy programs, reflecting traditional ways to 
boost persistence. Through the LILAA initiative, many of these existing approaches were en- 
hanced to better address problems of student persistence. 



Information-Gathering Strategies 

As planned, many of the LILAA programs’ information-gathering strategies are in- 
tended to inform the design and refinement of strategies in the other three categories. At the 
same time, many of these strategies were designed by the programs to provide ongoing sources 
of information to expand their knowledge, improve their persistence strategies, and monitor 
changes in outcomes. 

Early Formal Information-Gathering Methods 

Early in the project, all the programs employed some type of formal information- 
gathering methods (for example, surveys, case studies, workshops, interviews, and focus 
groups) to inform the design of persistence strategies. These efforts were led either by program 
staff or by consultants. Some program directors and staff felt that the process was a learning 
experience, and they appreciated the surprises that emerged. More often, however, they felt that 
the findings simply confirmed what they already suspected. For example, in a focus group at the 
Queens Borough Public Library’s program, staff and the persistence coordinator found that stu- 
dents felt they needed more emotional support (centering on issues of family and personal prob- 
lems), which reinforced an earlier decision to put some of the program’s resources into building 
a stronger sense of community. 

At most programs, the formal research efforts suggested too many needed strategies or 
strategies that were difficult for the programs to undertake. For example, it would be very diffi- 
cult for the New York Public Library’s LILAA program to develop transportation subsidies for 
its students that would sustain them over a long period of time. Such supports would be costly 
and might threaten the relationship with the host library, which believes strongly in the principle 
of equal access and was not accustomed to helping other low-income patrons with transporta- 
tion. Further, it would be difficult to develop criteria for establishing which students would be 
eligible for such subsidies. This program decided instead to ease transportation problems by 
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scheduling orientations at several branch libraries, rather than centralizing them. Other common 
concerns included the difficulty of collecting information from a representative group of stu- 
dents and the need to help students feel comfortable about revealing their problems and describ- 
ing their expectations. To address such concerns and reduce bias in gathering information, one 
program asked students hypothetical questions, and others guaranteed anonymity. 

Overall, few programs designed specific persistence strategies based directly on new in- 
formation that was gathered formally at the start of the LILAA initiative. For the most part, the 
process was informative but did not yield new knowledge leading programs to alter their ser- 
vices radically. A more defined “action research” process — with specific expectations for 
seeking in-depth information about persistence barriers and reflecting on what was learned — 
might have led to insights producing additional innovative strategies that targeted particular bar- 
riers. Instead, the programs typically initiated a set of persistence strategies based on their own 
or a consultant’s existing knowledge, while gleaning some ideas from student focus groups and 
from surveys that deepened their understanding of problems. Over time, staff “tinkered” with 
the strategies as they learned more. 

Upgrading Data Systems 

In addition to the formal information gathering at the start of the LILAA project, the 
LILAA programs also enhanced ongoing data collection processes and electronic data systems 
in order to better track students’ characteristics, attendance, goals, and test scores (in some 
cases). Though this upgrading was driven partly by the need for data for this study — data that 
have to be comparable to the National Reporting System (NRS) data that are collected by pro- 
grams funded by the Workforce Investment Act (WIA) — the library programs also benefit di- 
rectly by having more detailed information about their students over the life of the study and 
beyond. Also, having a more comprehensive data system encouraged some programs to collect 
additional data for their own purposes. Besides asking more questions during intake, some pro- 
grams also implemented additional literacy testing. For example, one library introduced admini- 
stration of the Test of Adult Basic Education (TABE). 

Although all nine of the library literacy programs had some type of data collection sys- 
tem in place prior to this study, most found it a challenge to collect more detailed and more fre- 
quent information from their students. The programs in the persistence study are small, and data 
systems and testing are expensive. Box 3.1 highlights issues encountered by the programs dur- 
ing the initial implementation phase."* 



"'The final report in 2003 will examine whether or not the programs resolve some of these issues. 
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Box 3.1 

Tracking Persistence: Challenges and Strategies 

With the LILAA programs serving approximately 50 to 200 students in any given 
month — all participating in a variety of activities — collecting, entering, and man- 
aging data on all these students is an enormous task. Following are some of the chal- 
lenges that the library literacy programs face in collecting and managing the data for 
the persistence study, along with some strategies that the programs have imple- 
mented to address these challenges. 

Maintaining Students’ Privacy. An important tenet in the U.S. public library sys- 
tem is privacy. Libraries have traditionally been places where members of the com- 
munity can go to learn, with relative anonymity. Therefore, collecting data on liter- 
acy students’ backgrounds and activities while in the library (or in a center run by 
the library) can seem to contradict libraries’ commitment to privacy. Although all in- 
formation collected by program staff is kept completely confidential, both by the li- 
braries and by the research team, some questions (such as employment status or 
hours spent working with a tutor or using a computer) are difficult to ask in a way 
that clearly respects students’ privacy. Sensitivity helps, but program staff recognize 
that students do not always accurately account for their activities and may refuse to 
answer questions about their experiences, as is their right. 

Securing Tutors’ Cooperation. For many of the programs, collecting participation 
data on pair-tutoring is an ongoing challenge. Particularly for tutoring that takes 
place off-site, the main challenge is checking with tutors to ensure that they are re- 
cording information and relaying it to program staff. Some tutors may be unwilling 
to provide these data for any number of reasons, perhaps because they want to pro- 
tect the privacy of their students, because they do not understand or agree with the 
importance of reporting the information, or because they stop volunteering with the 
program. Staff may try calling tutors to collect information that they have not re- 
ceived, which is very time-consuming, or they may try to collect the information di- 
rectly from the students. No programs currently have systems in place that stream- 
line the process of gathering information from multiple informants, but some have 
suggested or tried implementing a number of approaches to improve their chances of 
receiving data from tutors. Some send monthly E-mail reminders to tutors. Others 
have introduced incentives, such as gift certificates or recognition awards. Many 
programs also make an effort to ihform tutors about the importance of data collec- 
tion and of this evaluation. 

(continued) 
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Box 3.1 (comtimued) 



Capturing Participation in Independent Activities. In addition to structured liter- 
acy instruction, all the LILAA programs offer independent activities such as com- 
puter instmction, library research, or book borrowing, and capturing these activities 
is important for a comprehensive assessment of student persistence. However, aside 
fiom computer instmction, few independent activities are captured consistently. This 
means that persistence is consistently underestimated, both because students may 
continue to engage in literacy learning after they stop participating in tutoring or 
classes and because students likely spend many more hours working on literacy 
skills than the few hours a week they typically spend with a tutor or in a small-group. 
Programs have implemented a number of approaches to improve their assessment of 
participation in independent literacy activities. Most of these rely on students’ self- 
reports. Programs encourage students to self-report all their literacy activities, and 
they make doing so as easy as possible. For example, several programs leave a sign- 
in/out sheet, have a punch-card system, or have installed sign-in/out software on the 
computers. 

Upgrading Technological Capacity. The LILAA literacy programs entered the 
persistence study with a wide range of experience in using data management sys- 
tems, but they all faced challenges in upgrading their processes, keeping them mn- 
ning smoothly over time, and ensuring that the data were sufficiently accurate and 
timely for the study. Besides getting the required hardware, they needed to learn how 
to set up and use the software to capture program activities. For some of the pro- 
grams, this study’s data collection needs coincided with other efforts to collect and 
report data for other funders or for internal uses, and all programs seemed to benefit 
fiom the new data management requirements and support. Staff at one of the two 
programs that developed databases fiom scratch commented that the new systems 
helped them leam who their students are. Other programs that were already using 
these systems were able to expand their use, moving beyond reports of program 
status to using the software for case management and other internal functions. 

Streamlining Data Entry. Entering all the demographic, attendance, and other pro- 
gram data into a database can be very time-consuming. For busy staff who are dedi- 
cated to providing literacy services to students, the task of entering data for all stu- 
dents every month was an additional burden. In this regard, some database software 
was much more time-consuming than others. To ease the time burden of data entry, 
some programs designated a staff member to serve as a part-time data manager. 
Other programs designated a time for data entry — setting aside one day a month, 
for example, to enter all data collected on paper forms. The strategy of designating a 
person or a time for data entry appears to prevent severe backlogs from developing 
and produces data that are more useful and more reliable both for the programs and 
for the study. 
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Participating in Site Visits and Conferences 

All the LILAA programs took part in site visits to other LILAA programs — visits that 
centered on technical assistance and were arranged either independently or through the Illinois 
Literacy Resource and Development Center, an organization that provides technical assistance 
to the project. These visits allowed the staff from the host and the visiting sites to get to know 
one another better, and they provided concrete experiences of actually seeing another program 
and appreciating its different approaches. Program staff from each site also attended confer- 
ences for all LILAA grantees. These meetings were important opportunities for site staff to 
learn directly — through observation, discussion, and exchanges of materials — about each 
other’s efforts to improve student persistence. For example, after a site visit to Oakland, the 
Greensboro program adopted Second Start’s data system and decided to hire someone to man- 
age its database, as is done in Oakland. Similarly, Redwood City’s Project READ redesigned its 
student forms based on other programs’ materials. 

Unfortunately, some responses to these meetings also show their limits. The examples 
above of staff who reported learning from their peers at visits and conferences are offset by re- 
sponses from staff who said that, after being exposed to other programs at the first conference, 
they did not learn much more from later gatherings or from the listservs set up to supplement 
these meetings. Some staff also felt that their own program’s needs were unique, which limited 
how much they could learn from each other. A different design for the site visits and confer- 
ences — organized, for example, around specific issues and common problems — might have 
reduced the amount of show-and-tell and led to a more coherent professional dialogue that fo- 
cused on problem-solving, continuing education, building coalitions, and financial support for 
the group. On the other hand, LILAA program staff did also say that the conferences focused 
attention on student persistence and helped create a sense of being in a collegial community. In 
addition, many staff attended national conferences that were highly beneficial to them, and they 
also participated in national listservs. 



Instructional Strategies 

Because literacy instmction is the mission of the LILAA programs, the libraries in flie 
persistence study introduced innovations to improve instmction and also developed strategies to 
align their instmction with students’ needs. The seven most common strategies found in the 
LILAA programs are described below. 

Improving Student Orientations 

When potential students first approach a library literacy program, they typically receive 
some form of orientation, either individually or in groups. Most orientations last an hour or two 
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and include a discussion of the program and a tour of its fecilities. The sessions usually follow 
an intake and assessment procedure in which students provide information about themselves 
and explore their goals. Efforts to improve orientations often seek to reach out to a more diverse 
population, to introduce a stronger sense of community, and to provide a better picture of what 
to expect from the program. For example, the Glenwood program added an orientation in Span- 
ish in order to better reach Greensboro’s Hispanic community, and New York Public Library 
created a new set of orientation sessions to teach students basic academic skills and prepare 
them to enter a tutoring group. Other improvements address issues of scheduling and accom- 
modating students who are waiting for tutors, in order to minimize the time between showing 
interest in the program and being able to attend an orientation. Project READ, for example, ex- 
panded its programming to accommodate new students and launched an intensive effort to re- 
duce its waiting list. Better orientations might encourage more potential students to enroll in 
services or to start studying with more enthusiasm and preparation. At the same time, if orienta- 
tion provides a realistic view of instruction, some potential students might decide that it is not a 
good time to begin studying or that the program does not fit their needs well. 

Improving Classes and Group Tutoring 

Most of the library literacy programs began by providing “pair tutoring” (individual in- 
struction by a volunteer tutor) and continued this service consistently throughout the LILAA 
initiative. As a strategy to address low student persistence, all the programs also begun adding 
classes and small-group tutoring; some did so even before the persistence study began. These 
options not only allow programs to serve more students but also add an element of group inter- 
action that is missing from pair tutoring and computer labs, which are more individualized. 
Group interaction may support persistence, since most students value the social exchange. In 
addition, when tutors miss scheduled sessions, students can study with each other. Group tutor- 
ing often takes place on-site (in contrast to much pair tutoring, which can occur away from the 
program), and so staff can monitor instruction and provide technical assistance to tutors imme- 
diately, if the need arises. The classes also create opportunities to use professional teachers and 
to focus on students’ particular needs or interests, such as spelling or poetry. 

Improving Tutor Training and Recruitment 

All the programs have sought to improve tutor training as a result of the LILAA initia- 
tive. The improvements include longer training, linking tutor training to student orientation, de- 
veloping or improving training manuals, and focusing on technology and greater sensitivity to 
cultural issues and -students’ goals. Programs have also provided more tutor support. New York 
Public Library’s tutor training was completely revamped through a participatory staff process, 
leading to standardization of information and materials across sites, centralizing locations and 
scheduling for training, formalizing and better organizing tutor intake and procedures, extend- 
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ing the hours and duration of tutor training, and raising the quality of instruction. The trainings 
have become more structured and systematic, and tutors now have new training manuals. Also, 
during the information-gathering activities at the start of the project, programs surveyed tutors 
to find out what support they needed. This led to more technical content in the training, such as 
how to teach reading and writing or how to support conversation and integrate cultural issues 
when working with ESOL students. Since tutors are being asked to report more accurately on 
the activities in their groups or one-on-one sessions, some of the added training also focuses on 
administrative matters. Programs are stricter about attendance at tutor training, and the training 
is now used to improve tutor selection and their commitment to the program. 

Longer and better-designed tutor training supports student persistence by improving the 
experience and the relationships that students have in the program. Staff believe that improved 
training can support tutor persistence as well — which is another critical factor in student per- 
sistence. The success of tutor training depends on the volunteers who come forward to be 
trained and on their motivation and willingness to learn fi’om the staff on an ongoing basis. Dur- 
ing the initial implementation phase of the persistence study, all the programs have also been 
experimenting with ways to recruit tutors who are more like their students or who are highly 
sensitized to any ethnic or linguistic issues that the students may have. One program recruits 
college students who are Spanish majors and are interested in Latino culture. 

Standardizing Instruction 

As part of the effort to improve literacy instmction, all the programs in the T.TT.A A persis- 
tence study have worked to standardize instmctional services within their library system. Stan- 
dardization can improve the services by ensuring that every student’s instruction is similar in con- 
tent and quality. Further, for programs that have multiple sites, standardization should make it eas- 
ier for students to move fi’om one site to another. Moreover, standardization can provide some 
economies of scale related to tutor training and the development of program materials. 

Documenting and Tracking Students’ Goals 

Before the LILAA initiative, literacy students’ goals were assessed only informally, if 
at all. All the library programs now ask students about their goals at intake, and these goals are 
added to a program’s data system. This is an important strategy for improving student persis- 
tence, because it helps the programs meet students’ needs and it gives students the chance to 
develop a plan against which progress can be measured. In the initial implementation phase, 
some programs were able to directly connect students’ goals and assessments. For example, the 
Queens Borough Public Library’s Adult Learning Centers used a participatory staff process to 
created a system of benchmark goals that focus on the development of reading skills. Most sites, 
however, do not yet have a clear system for linking instmctional choices to goals; during this 
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stage of implementation, most of the programs were establishing goal-tracking and were not yet 
focused on linking students’ goals to the curriculum. 

One example of a goal-setting strategy — which evolved out of collaboration at a 
LILAA conference — is the adoption of a form used to document students’ goals during the 
intake process. The form lists examples of academic goals (such as getting a GED or reading 
better) and of life goals (such as reading to a child or passing a driving test). The Glenwood 
program is helping students develop a “learning contract,” which lists expectations about the 
time and energy that a tutor can give to help the student reach stated goals within the boundaries 
of their formal relationship. Goal-setting is discussed further in Chapter 4. 

Addressing Learning Disabiiities 

If a literacy program is not sufficiently equipped from the outset to help students who 
have learning disabilities, the program is unlikely to retain those students. Therefore, instmc- 
tional strategies that are designed to identify and serve students who have learning disabilities 
are essential to improving persistence. One program. Project READ, hired a consultant who 
developed a programmed instmctional system to help tutors teach adult students who have 
learning disabilities. The consultant provided materials and training, and the program has as- 
signed one staff member to be a Special Needs coordinator, who works with the consultant to 
identify students who are considered “learning-disabled.” Other programs spent the year decid- 
ing whether or not to tailor instmction to this population and, if so, which were the best, most 
practical ways to implement specialized instmction and curricula. 

The data collected for the LILAA achievement study (see Appendix C) confirm that 
most of the English-speaking literacy students have very low reading scores and might benefit 
from programs that focus especially on learning disabilities. During the initial implementation 
phase, only Project READ addressed this need in an organized way, by designating a cadre of 
its better tutors and literacy instmctors to serve learning-disabled students. This approach might 
show the way for other programs that rely on volunteers. 

Administering Student Assessments 

More useful and regular assessment of student progress is a way to encourage students 
and to focus instmction. New York Public Library and Project READ have turned to norm- and 
criterion-referenced tests, such as the Test of Adult Basic Education (TABE) and other assess- 
ment instmments like the Bader Reading and Language Inventory,’ which are intended to be 



’This is a flexible battery of tests designed to quickly screen children, teens, and adult students in a 
nonthreatening manner. It was developed by Lois A. Bader at Michigan State University and is published 
by Prentice-Hall. 
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used at regular intervals and to supplement placement tests like the Reading Evaluation Adult 
Diagnosis (READ) test, or other types of informal assessments that are usually given at intake. 
Many of the programs try to reassess the students within designated periods (every six months, 
for example), which has proved to be worthwhile but is also labor intensive for staff. Second 
Start hired a part-time consultant to reassess students. One aspect of the LILAA research effort 
— the achievement study (Appendix C) — may also provide new ideas for strengthening as- 
sessment; it takes a reading components approach that focuses on students’ strengths and weak- 
nesses and provides useful information for instmction. 



Operational Strategies 

The LILAA programs focused on operational strategies intended to create more oppor- 
tunities for increased student participation. These were perceived to be an important support for 
persistence, especially for students who could only attend at certain times, and to provide in- 
creased access to diverse instmctional modes, like computer-assisted study. Two types of opera- 
tional strategies are discussed here. 

Extending of Hours of Operation 

The information-gathering activities undertaken by the programs during the planning 
phase of the project identified the need for more or different hours of operation, and some 
LILAA programs extended their hours in the first year. For example. New York Public Li- 
brary’s Fordham branch extended morning offerings in order to better serve mothers whose 
children are in school. Oakland Public Library’s Second Start program also extended its eve- 
ning hours, to give students greater access to the computer lab. Greensboro Public Library’s 
Glenwood branch created a “staggered hour” system to serve students who work in shifts. On 
the other hand. Queens Borough Public Library considered limiting its hours as part of a move 
to serve students better with existing resources and to help build community among both stu- 
dents and tutors by creating common times and a common space to gather. 

Expanding and Upgrading Computer Labs 

All the LILAA programs expanded and upgraded their computer labs during the former 
phase of the project (before 1999) and have continued to do that during the current phase. Addi- 
tional access to computers is considered a support to student persistence because computers also 
allow for self-study when instmction and tutors are not available. Moreover, many students are 
interested in learning about computers. The programs’ computer labs range in size fiom 5 to 20 
workstations, but students often have access to computers in the libraries’ main facilities as well. 
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Acquiring more computers and installing new software has been the easy part of lab 
expansion. There have also been a number of related challenges; for example, some programs 
do not have anyone on staff who has technical expertise. The programs usually have to train 
staff and tutors to use the computers and to integrate them into instmction plans that serve stu- 
dents’ needs. Some of the labs are staffed with paid technical assistants or college mentors, and 
students often help each other. The programs now have enough experience that they can de- 
velop a vision and plans to integrate computers into their services and instmction. 



Support Strategies 

The T.TT.A A programs’ staffs are well aware of literacy students’ many barriers to per- 
sistence, and they believe that it is essential to develop effective support strategies for adult 
learners. But most also see limitations on their ability to provide such services, and some ques- 
tion even whether they should attempt to do so, given their position as part of a public library. 
Broader library concerns about potential liability in the event of problems and about equity 
among all library patrons constrain the variety and scope of the services that they can provide. 
Many programs are more comfortable offering students referrals to other organizations in the 
community. The LEl/AA programs tend to follow the model of most K-12 and postsecondary 
institutions, which provide some limited nonacademic support but typically do so informally. 
This section briefly discusses strategies for offering literacy students referrals to support ser- 
vices and for providing direct, in-house assistance. 

Increasing Referrals for Assistance 

Most of the LE1,AA programs provide adult learners with some type of referrals to gov- 
ernment or community agencies that offer services relating to welfare, criminal justice, food, 
health care, and substance abuse. Some programs have developed collaborative relationships 
with social service agencies in the communities they serve, and they often receive reciprocal 
referrals. In response to the LILAA initiative, all the programs are looking for ways to build a 
larger network of service providers to help their students persist. Second Start has a relationship 
with an agency that refers potential students who are in the criminal justice system, but this type 
of relationship is uncommon among the referral networks of the LILAA programs. Most refer- 
rals take the form of a name, phone number, and address — which is often all that staff can pro- 
vide. During the first year, the persistence coordinator for the Queens Borough Public Library 
Adult Learning Centers started to design a system that will allow site directors to refer students 
to agencies that can help them with support services. 
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Offering In-Hoyse Assistance 

Although the LILAA programs rely heavily on referrals to address students’ needs for 
support services, they also provide some forms of direct support that address barriers to persis- 
tence. These services include food, transportation subsidies, child care, and informal counseling. 
For example. Second Start provides students with bus tokens or passes. Greensboro developed a 
process for determining transportation needs, and eligible students receive bus tokens only if 
they request them. 



Conclusion 

In the initial implementation phase of the LILAA initiative, the nine library literacy pro- 
grams in the persistence study gained a new focus on the persistence of adult learners. All the 
programs embarked on planning and implementing a variety of strategies. But as the year 
progressed, they found that the heightened emphasis on student persistence was the most sig- 
nificant innovation in the LHAA project. As the programs gathered information on barriers to 
persistence and devised and implemented strategies to overcome them, they encountered a vari- 
ety of challenges. The challenges became learning opportunities, however, and allowed the pro- 
grams to better understand the varying and complex needs of their diverse student populations 
and to assess the feasibility of providing certain services, particularly support services. 

Most important, perhaps, is that the programs gained a better overall connection with 
their students and came to recognize that persistence issues make an effective lens through 
which to discover and examine program enhancements on multiple levels. Moreover, as a result 
of the LILAA initiative and the heightened desire to track and understand student persistence, 
the programs placed particular attention on improving their data systems. The enormity of this 
task also presented new challenges, but the programs succeeded in establishing or upgrading 
valuable data systems that track students’ participation and literacy activities more precisely 
than ever and that provided the data for the analyses in Chapter 2. (Appendix C describes some 
of the data issues related to the persistence study.) 
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Chapter 4 



What Are the Early Lessons 
from the LILAA Persistence Study? 

Since the launch of the Literacy in Libraries Across America (LILAA) initiative in 
January 2000, the LILAA programs in this study have heightened their focus on student persis- 
tence and have planned and implemented strategies to improve persistence, either by enhancing 
program elements already in place or by introducing new processes or systems. Chapter 3 re- 
viewed the progress that the programs have made in applying these strategies, and this chapter 
explores early lessons emerging from their experience and suggests additional steps that library 
literacy programs could take to support persistence among adult literacy students.' 

Three main areas have emerged for focused attention: personal goals, sponsorship, and 
respect and care for adult students. This chapter discusses the linkage of each factor to persis- 
tence, gives an overview of what the LILAA programs are already doing in these areas, and pre- 
sents some early lessons for improving student persistence. 



Personal Goals 

Some adult learners persist in a literacy program because they enjoy the experience, but 
many do so because they are driven by personal goals, which can take many forms. The litera- 
ture on persistence suggests that literacy programs’ efforts to assist students in defining and 
achieving goals are essential for supporting persistence. 

Students’ goals tend to fall into two categories: instmmental and transformational. In- 
strumental goals typically indicate specific outcomes that are essential for achieving longer- 
term aspirations. Examples include passing the General Educational Development (GED) exam, 
getting a driver’s license, or being able to read job-related materials, such as blueprints or build- 
ing codes. Instrumental goals may also involve incremental objectives — such as learning 
phonics, vocabulary, or the alphabet — within a broader goal of improving literacy skills. At- 
taining an instmmental goal often leads to the development of a new goal, which encourages 
ongoing participation in the program and, therefore, persistence. 

Transformational goals entail broader changes that students want to achieve, such as 
changes in self-perceptions or identity, major life skills, psychological states, and social or work 



'This discussion is based on in-depth interviews with students, tutors, and staff as well as a review of 
the literature on adult student persistence. 
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roles. These tend to be intrinsic to the student and are described as enhancing the quality of the 
student’s life on a deeply personal level. During interviews, many literacy students told stories of 
major events in their lives that propelled them toward changing. Some had overcome substance 
abuse, left abusive relationships, or immigrated to the United States. Others hit an employment 
barrier because of low literacy skills. Such pivotal events created a desire for self-improvement, 
which led to enrollment in a literacy program. Many students in the study said that they were 
transforming themselves fi'om who they had been to who they want to become, and they reported 
that they viewed the library literacy program as part of this transition. These students came to the 
library, in part, to find a community that would support their new identities. 

Although some students’ goals may be considered unrealistic by program staff, the stu- 
dents may actually be describing a transformational goal. For example, students whose literacy 
skills are extremely low may state that their goal is to pass the GED exam. For some, the desire 
to pass this test may, in fact, be a transformational goal that expresses a desire to change identity 
— to become a person who has a GED and therefore is “educated.” 

Mixing Instrumental and Transformational Goals 

Elena. Elena’s immediate goal is to pass a licensing exam that is required in 
order to become a pharmacists’ assistant, and her tutor is helping her build 
basic skills and prepare for the exam. Elena’s long-term dream is to become 
a lawyer who defends womeri who are trying to leave an abusive relation- 
ship, just as she did in an earlier stage of her life. Both of Elena’s goals — 
the first more instmmental, and the latter more transformational — motivate 
her to persist in literacy studies. 

Joe. When Joe entered the literacy program, he just wanted to improve his 
reading and writing. Now he is reading romance stories, surfing the Internet for 
sites about wrestling, and writing autobiographical essays. Being able to handle 
all these materials signifies that Joe has achieved his first instmmental goals. 

After working closely with his group and tutor, he decided that he wanted to 
get a driver’s license and a GED certificate, but he was nervous about passing 
the two tests. In the course of his studies, Joe became more aware of his grow- 
ing dreams and needs, including learning to read and write, getting an educa- 
tion, and, someday, having a family. He said: “I just want to be proud of more 
things for myself, and — family. That will make me proud.” 
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Efforts to Support Goal-Setting at the LSLAA Programs 

Most students in the persistence study said that they were asked about goals when they 
entered their T.n.A A program, and they appreciated the informal discussions with tutors and 
staff about their goals and the steps to attain them. However, some students had difficulty ar- 
ticulating their goals. Others, who were in the middle of a crisis, found it difficult even to think 
about goals. To help students consider and set personal goals, program staff interviewed stu- 
dents using a checklist of goals that are in line with the mission of the initiative. Also, as men- 
tioned in Chapter 3, program staff and tutors did not have models for how to link goals directly 
to instmction. Though some tutors did achieve a link (especially in one-on-one tutoring), most 
tutors need training and technical assistance to accomplish this. 

Lessons About Personal Goals and Student Persistence 

Besides the strategies that the LILAA progr^s already have in place, the following are 
early lessons about'focusing more on students’ goals and luiking them to increases in persistence. 

• Encourage an ongoing, reflective dialogue with students about their goals, 
starting at intake and continuing throughout program participation. 

An ongoing dialogue about goals might help students keep the source of their motivation 
in mind and, therefore, might promote greater persistence. Frequent discussions about goals also 
send a message that staff and tutors are interested in seeing students achieve their goals. A discus- 
sion of goals should be sensitive to the feelings of the student and should allow for the identifica- 
tion of both instrumental and transformational goals. Making sure that students feel comfortable in 
exploring their goals could also support persistence. Library literacy programs could view them- 
selves as what the Adult Development research team refers to as “holding environments”: suppor- 
tive, stable places that are available to the students at any stage in their learning and that also rec- 
ognize their goals for change.^ Such an environment might help students make the transition to the 
next stage in their lives. Further, programs could promote peer interaction, thus providing students 
with support by being among others who are making similar life changes. 

Staff and tutors could also encourage students to reflect more on their goals rather than 
merely naming them — as a way to develop a deeper sense of meamng. Goal discussions might 
be more productive if the students begin with an examination of their lives and dreams, rather than 
with a list of potential goals. The list then might be a good second step in this dialogue. Discus- 
sions among groups of students can help broaden and deepen an initial understanding of goals and 
can show students that others may be pursuing similar goals. For students whose goals are not yet 
clear, instmction could focus on their interests and lives. Dialogue journals, reflective discussions. 



^Kegan et al., 2001. 
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wntten personal histories, and the reading of student stories can also support the process of goal 
clarification. Most tutors and staff need training to coach students through such conversations. 



Linking Instruction to Goals 

If the ongoing discussion of goals leads to changes in instruction, students’ motivation 
may become stronger, particularly if they see progress toward achieving their goals. Tutors can 
be trained to teach skills within the context of any instrumental or transformational goal. For 
example, a class about a particular skill that is often an instrumental goal for students can also 
reveal a path for progress. In terms of transformational goals, some tutors already have students 
reading materials fi'om home, presenting their opinions, stating their feelings, and exchanging 
ideas about issues. These approaches to get students more intensively engaged in their learning 
could lead to a stronger commitment to the program and could improve persistence. 



Sponsors and Sponsorship 

Many potential students are unaware of the educational opportunities that exist in their 
communities, or they may not think of education as a way to make positive changes in their 
lives. Others may want to pursue an education but are intimidated by the institutional context of 
formal education. These potential students often depend on the help of a sponsor who knows 
more about educational opportunities and introduces them to a learning program. Sponsors also 
can play a key role in helping students persist in their education after they enter a program. They 
help students endure feelings of fiiistration, difficulties with self-study and homework, and 
changes in tutors. They also provide continued support to students who have to leave a program 
and may encourage them to return when they are able. Sponsors provide various kinds of en- 
couragement and help: material support (cash for lunch or transportation), emotional support 
(encouragement to attend the program and study every day), literacy support (document assis- 
tance), informational support (information about job certification programs), or symbolic sup- 
port (the memory of a parent who valued education and who instilled this value in the student). 

Literacy students have different types of sponsors, who may intervene on their own or 
be called upon by the student. Personal sponsors include relatives, partners, fiiends, and co- 
workers. Official sponsors are professionals who are assigned to provide institutional support to 
the student; they include social workers, parole officers, welfare-to-work counselors, profes- 
sional literacy staff, libranans, and teachers. Intermediate sponsors are associated with a social, 
religious, or educational organization but are in a less formal relationship with the student; ex- 
amples include pastors, sponsors or fellow members of 12-step programs, volunteer tutors, and 
other students. 



- 60 - 




79 



A personal sponsor (for example, a relative who gives material, emotional, and infor- 
mational support) typically offers more pervasive and more comprehensive support than an of- 
ficial sponsor (such as a caseworker who provides a referral to a program), who gives intermit- 
tent and targeted support within a limited time fi:ame. Intermediate sponsors are in the middle of 
this continuum; they are involved with students for a longer period of time than official spon- 
sors, but they are not integrated into students’ lives as personal sponsors are. A student’s con- 
nection to an intermediate sponsor is not encumbered by the kinds of demands that fiiends and 
relatives make on each other or by the institutional objectives of official sponsors. 

The lack of personal demands along with the longer period of connection may make in- 
termediate sponsors particularly beneficial to improving student persistence. Intermediate spon- 
sors may become both fiiend and mentor, provide emotional and spiritual support, and help the 
student with everyday problems. Intermediate sponsors are less likely than official or personal 
sponsors to provide material support. Their relationships are more distant than those of personal 
sponsors, and they are not as cormected to resources as official sponsors often are. However, the 
intermediate sponsor may serve as an important role model for the student. 

Earlier LILAA reports identified support fiom family, fiiends, teachers, and fellow stu- 
dents as important to persistence.^ This report moves to the next step by suggesting that specific 
persons and groups can be key providers of certain forms of support. Sponsors can provide a 
link to social services, encouragement to continue studying, role models of success, emotional 
support, exposure to new ways of using literacy skills, and supplemental literacy instmction and 
practice. Sponsors support persistence at the program site and in the daily life of the student, 
serving as important but often-hidden keys to participation and persistence. 

Sponsors Come from Many Places 

Melissa. Melissa, a coworker in Rod’s family business, noticed that Rod had 
problems reading, and she offered to tutor him. Because she was one of the 
first people to talk with him about reading problems. Rod was more open 
about his literacy situation at work. Melissa also called Project READ and 
cormected him with the program. Melissa is a personal sponsor who is part of 
Rod’s life and who has shown that she is willing and able to support his per- 
sistence in literacy study by offering informational, literacy, and emotional 
support. 

Mark. When Mark was alone in the store where he worked, he was afiaid that 
he would make a mistake on the forms that he had to fill out. His wife encour- 



^Comings, Cuban, Bos, and Taylor, 2001; Comings, Parrella, and Soricone, 1999. 
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aged him to join the library literacy program. But Bill, his sponsor in Alcohol- 
ics Anonymous (AA), felt that Mark needed to reduce stress in his life and fo- 
cus on his drinking problem before taking on the task of improving his literacy 
skills. Bill encouraged Mark to go through one year of recovery and then reen- 
ter the literacy program. When Mark returned to the program, his tutor encour- 
aged him to use AA materials as a focus of study. Mark connects his persis- 
tence in the program with his persistence in AA: “I really want to stay here as 
long as it takes,” he says. “I want to be able to stick this out like I’m doing with 
AA.” Bill is an example of an intermediate sponsor, an individual who is part 
of a voluntary self-help organization. Though Bill is not a member of Mark’s 
family, he does have a long-term helping relationship with him through AA. 

Efforts to Build Sponsorship at the LILAA Programs 

Most of the literacy students who were interviewed for the persistence study mentioned 
a sponsor or several sponsors who made them aware of educational opportunities and who en- 
couraged and helped them to begin program participation. Though the LILAA programs do 
identify the roles of sponsors as important in supporting student persistence, they initially did 
not seem to be aware of their students’ sponsors, or — if they were aware — there was no for- 
mal process for reaching out to sponsors. Now, however, the programs view tutors as assuming 
some sponsorship roles, rather than being disengaged from students’ lives. Most programs seek 
tutors who are willing to support students’ learning efforts, to advocate for them as needed, and 
to provide instruction that is attuned to students’ needs. Some program staff also play sponsor- 
ship roles through informal advocacy, assistance in filling out forms, referrals to social services, 
and calling students to check on how they are doing. But, as is the case with some tutors, staff 
members often do not want to become deeply involved in students’ lives, out of concern for 
being intrusive. Similarly, some students do not want relationships that make them feel depend- 
ent, and, in some cases, sponsorship may discourage participation. 

Lessons About Sponsorship and Student Persistence 

• Promote sponsorship as an activity that fits easily into the mission of li- 
braries as part of a network of community-based services. 

Libraries have always promoted literacy and supported their patrons by providing 
community information, some direct assistance, and referrals."' Referrals may also help literacy 
students find individuals who may become sponsors. Moreover, students in literacy programs 

■'Estabrook and Lakner (2000) found that 90 percent of the libraries in their sample were referring po- 
tential student to literacy programs; this referral process is part of the institutional sponsorship role. 
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may have greater access to library-based supports, because of the equal access principle 
whereby public libraries try to stay open most days of the week and for long hours each day. 

Many libraries do not provide direct instmction in literacy but do provide referrals to lit- 
eracy programs. Given the importance of sponsorship in supporting student persistence — as 
demonstrated by the data emerging from the LILAA study — staff in such libraries could ex- 
pand their sponsorship efforts to include persistence support for students who are involved in 
self-study or in other literacy programs outside the library. Libraries that do provide direct liter- 
acy instmction might extend their sponsorship role to students who study in outside programs, 
which might have greater difficulty attracting sponsors. 

One challenge in having sponsors link students with social services is the need to keep 
referral information and contacts current and relevant. Libraries could enlist knowledgeable 
community members to volunteer their time and to work with reference librarians to expand 
their referral services. Some volunteers might also act as sponsors for adult literacy students. 

Libraries could also provide training and support to volunteers who play sponsorship 
roles. Sponsorship could well be seen as less of a commitment than tutoring; some might find 
it less daunting to provide “moral support” than direct literacy instmction. More library pa- 
trons might volunteer to assist the program in this way, and some might eventually become 
tutors. Volunteer sponsors could facilitate reading activities, help with homework and self- 
study, and assist in using library resources for information and referrals to social services. 
Some of the activities that libraries now undertake to encourage children to improve their lan- 
guage skills could be adapted for adults; creative staff might find new ways to provide diverse 
types of support for adult learners. 

• Reach out to all types of sponsors, and involve them in the program. 

Library literacy programs could do more to learn about their students’ sponsors and in- 
volve them in the program. Rather than relying on casual chats during intake, assessment, and 
classes, program staff could question new students about their sponsors and could help students 
develop strategies to engage sponsors in helping them persist in their studies. Programs could 
involve some sponsors directly in services and could provide training to help sponsors expand 
their support of students. 

Because most sponsors are part of social networks outside the library, formal links be- 
tween the literacy program and these networks might spread the sponsorship role into more parts 
of students’ lives and might bring more people from these networks into the program. A partner- 
ship among sponsors, students, tutors, and staff might make the program experience more direct 
for students, which could increase their chances of persisting — and of achieving their goals. 
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• Expand staff members’ roles in sponsorship. 

Some literacy program staff and tutors are already playing sponsorship roles for stu- 
dents, but a conscious effort to expand such activities might develop new ways to support adult 
learners. For example, some program staff — who are often former or current students — might 
call inactive students who have dropped out of the program to find out why they left and en- 
courage them to come back. Staff and tutors might make it a point to talk with active students 
regularly, as a way to encourage them to persist. Developing a variety of peer support methods 
and opportunities is another way to strengthen the program’s sponsorship roles. 

Although many literacy programs do connect students to services they need, the staff 
might develop strategies to identify what kind of support services are needed, rather than wait- 
ing for students to come forward and seek help. A proactive approach would require staff to be 
trained in counseling and group dynamics, in order to provide safe opportunities for students to 
share their problems. Alternatively, a social service agency could play this assessment role, in 
collaboration with program staff 

Respect and Care for Adult Students 

The principles that guide library services emphasize respect and care for adult students. 
Although this common quality may not “look” the same in every library literacy program, it de- 
fines their approach. Library literacy programs mostly serve students who are below the fifth- 
grade reading level. In fact, some students come to library programs after state- and federally 
funded programs rejected them because their literacy skills were too low. These and other students 
may feel isolated and marginalized in their communities, may be disabled, or may have histories 
of abuse or trauma. Such students are often the hardest-to-reach populations in a community, and 
they are more likely to respond well to informal, small learning environments. For students who 
fail in formal schooling, the libraries provide an opportunity to learn in a less threatening envi- 
ronment. The more comfortable students feel, the more likely they are to persist. 

Respecting Students’ Needs 

Nan. Nan receives a lot of help fi'om her tutor, whom she acknowledged with 
a letter of thanks in a book of student writing. When she has problems with 
her husband. Nan feels comfortable talking with a female staff member, who 
listens carefully and helps her resolve family conflicts. Sometimes, when a 
baby-sitter is not available, the staff take care of her 6-year-old. When Nan 
leaves for home after the program twice a week, she says that she feels “like 
a million dollars.” 
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Amy. Amy’s tutor, Mary, brings romance novels to their tutoring sessions at 
Amy’s apartment. Mary knows that these books will be interesting to Amy 
because the two women have had deep personal conversations. Amy is often 
homebound and sometimes fearful of leaving her house, but Mary gives her 
constant encouragement to be assertive and try new things. The student and 
tutor compare their life stories and even talk about their marriages. 

Respect and Care for Students at the LILAA Programs 

The nine library literacy programs that are part of the LILAA persistence study treat 
their adult students with respect and attempt to provide caring, personalized environments for 
them. Many students expressed their appreciation for this attitude, and some cited it as contrast- 
ing with their experiences in programs at community colleges and local education agencies, 
which they viewed as more administrative-oriented and less personal. The LILAA programs try 
to represent the cultures of the people they serve. All the programs have reading materials and 
decorations that reflect student cultures, as well as some staff and tutors who share those cul- 
tures. Students are given personal recognition through learning celebrations and booklets that 
feature their writings. The LILAA programs offer an opportunity for all students to feel socially 
integrated into a mainstream institution — sometimes for the first time in their lives. 

Students in the LILAA programs take part in open discussions about their ideas and 
feelings with tutors, other students, and program staff. These conversations take place while 
working in the computer labs, reading their own writing to tutors and other students, and ex- 
pressing their opinions about what they are reading in a class. Students encourage one another 
to speak in classes. This level of dialogue is much greater than in most literacy programs, which 
are focused more on skill development. Many students expressed in their writing and in inter- 
views that this program made them feel for the first time that they “came first” within a main- 
stream institution. 

The ULAA programs appear to have a strong commitment and an ability to provide a 
positive environment and sense of community, which should be supportive to student persistence. 
Yet even a positive support can have a negative effect as well. Staff mentioned, for example, that 
some students have been in the program for a long time but have made little progress. These stu- 
dents feel at home in the program, and they persist because they enjoy being part of this commu- 
nity and its social activities; but they may not be focusing enough on literacy goals. 

It is not always easy to maintain a positive learning environment. Some tutors become 
more involved in students’ lives than they want to, and so they stop tutoring. Some tutors are 
not culturally sensitive and may offend their students. The fi'ee expression that students are en- 
couraged to demonstrate in the programs may conflict with library policy, since libraries value a 
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quiet atmosphere. Even with such limitations, however, the environment of these library literacy 
programs is usually a positive support to student persistence. 

Lessons About Community and Student Persistence 

• Cultivate a strong sense of community for all students. 

The persistence of literacy students is enhanced when they feel that their program ac- 
knowledges and respects their affective sides and their cultures. The library programs might 
consider formal staff training in diversity issues and communication skills to ensure an even 
more supportive and accepting environment. A safe space that offers continuous care and re- 
spect makes students feel more comfortable about expressing themselves and pursuing their 
goals, and they will keep coming back. This kind of environment is especially important for 
students who have very low literacy levels. 



Lessons for Other Libraries 

The foundation for improving student persistence in a library literacy program is a posi- 
tive relationship between the program and students. This chapter explores three supports that build 
such a foundation when they focus instmction on the personal goals of students, promote a spon- 
sorship network for students, and treat students with care and respect. These early lessons fiom the 
LILAA persistence study provide insights into how libraries can connect to students in ways that 
support participation and persistence in learning, even among literacy students who are not in li- 
brary programs. In fact, libraries have features — such as a community identity, equal access poli- 
cies, and an institutional philosophy of accommodating the personal needs of patrons — that 
make them good venues to support the persistence of a variety of community members. The 
LILAA project will continue to explore the strengths of the libraries in the persistence study and 
will identify additional strengths that can help build or improve persistence in a literacy program. 



What’s Next in the LILAA Persistence Study? 

The findings of this interim report set the stage for further analysis to be presented in a 
final report scheduled for the fall of 2003. That report will focus on five questions: 

1 . What are the key factors supporting and inhibiting participation in adult lit- 
eracy programs? What are the implications of these factors for the design of 
strategies to improve learner persistence? 

2. Over time, how did the sites in the persistence study change their operations 
to support student persistence more effectively? What innovations were 
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strengthened or put in place during the second and third year of the initiative? 

What operational lessons emerged as programs worked to support improved 
persistence? 

3. As the reforms took effect, did student persistence improve over time? Did 
learners who entered the Ltt-AA programs during this later period persist 
longer or participate more intensely? Were there differences among types of 
students? Were there differences across the sites? 

4. Are there types of innovations that appear to be especially promising as 
ways to improve student persistence? What efforts and resources were 
needed to put these innovations in place? What are the implications for fu- 
ture program design and operations? 

5. What is the relationship between participation in library literacy services 
and improved literacy skills? What were the gains in literacy achievement 
test scores between the initial testing of students and a follow-up test? Did 
students who participated more in servaces show greater gains? Were there 
types of students who showed especially strong gains? Who showed little or 
no gains? What are the implications for program design? 

The upcoming research will also explore how library literacy programs fit within the broader 
adult education system. 

Each of these questions breaks new ground for program operations and research on 
adult literacy. The answers will help program operators and funders more effectively serve 
adults who face serious barriers to full participation in the nation’s economy and civic life. 
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Appendix A 



The Five Libraries Participating 
in the LILAA Persistence Study 



The five libraries in the LILAA persistence study illustrate a portion of the diversity of the 
library literacy programs in the United States, both in setting and in the students they serve. Each 
library has shown organizational stability and has leaders who are interested in making changes to 
enhance student persistence. These programs are not intended to be a representative sample of all 
library literacy programs, but they instead can illustrate what happens when experienced programs 
make persistence a focus, receive special support to enhance services, and are part of a special 
effort to monitor progress and assess effects on students. The following brief descriptions provide 
background information about the study sites and the variety of settings, services, and students. 
Some of the five libraries have multiple literacy programs operating out of branch libraries, and 
these are included in this research, bringing the total to nine literacy programs.' 



Greensboro Pyblic Library 

The Greensboro Public Library’s literacy programs evolved fi’om a community effort to 
address Ae needs of adults with low literacy skills. This resulted in Literacy 2000, a collabora- 
tive plan supported by the library, Guilford Technical Community College, and Reading Con- 
nections (a nonprofit agency supporting one-on-one tutoring), and other community agencies. 
The literacy programs that were started through Literacy 2000 are located at two of Greensboro 
Public Library’s nine branches, one of which (Glenwood) relies on Americorps volunteers for 
many of its staff members. 

9 Chavis. Also known as the Lifelong Learning Branch, the Chavis branch of- 
fers adult students afternoon and evening classes in preparation for the Gen- 
eral Educational Development (GED) exam, taught by teachers fixrm the lo- 
cal community college. The program also has a computer lab and gives in- 
struction in word-processing and E-mail. 

0 Glenwood. The Glenwood branch is in a working-class neighborhood that 
has attracted many refugees and immigrants fix)m all over the world. The li- 
brary is housed in a modem, sunny building that is a source of pride to the 
community. It offers small-group instruction in English for Speakers of Other 
Languages (ESOL), family literacy classes, a computer lab, and a collection 
of foreign language and multicultural reading materials. 



'The five libraries use somewhat different terminology in naming their overall local LILAA effort 
and the activities at specific branch libraries. For ease of exposition, this report uses the label “library” to 
identify the five LILAA grantees in the persistence study, and it uses the label “program” to identify spe- 
cific sites where literacy services are delivered. 
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New York PublSc Library 

New York Public Library (NYPL) has a long history of providing adult literacy ser- 
vices. Since the nineteenth century, NYPL branches have helped immigrants assimilate into 
the local community. Immigrants still come to the NYPL branches for English instmction, 
preparation to pass the citizenship test, and reading materials in their own language. NYPL 
provides literacy services through the Centers for Reading and Writing (CRWs) at 9 of its 85 
branches. The CRWs — funded primarily by the New York City Adult Literacy Initiative 
(NYCALI) — target adults at the lowest levels of literacy. Although the CRWs still serve 
many new immigrants, the focus of NYCALI is currently on literacy for English-speaking 
adults. The wide array of ethnic groups and nationalities served by the CRWs is reflected in 
the diversity of program staff and in the learning materials. The LILAA persistence study is 
focused on three of the CRWs. 

• Fordham. The Fordham branch is in a densely populated, thriving business 
district in the Bronx and houses the CRW in several rooms in the back of the 
library. The program serves approximately 150 students at any given time. 

The students participate in small-group instmction in the evenings and/or in 
tutoring sessions held throughout the day — most of which are led by volun- 
teers. Students also have access to a computer lab, where they can use the 
Internet, E-mail, and independently work on literacy activities and educa- 
tional software. 

• Wakefield. Housing one of NYPL’ s oldest CRWs, the Wakefield branch is 
located in a residential neighborhood also in the Bronx. The program serves 
approximately 100 students — mostly Afro-Caribbean adults — many of 
whom live within walking distance of the library. Students participate in 
small-group sessions and in computer study. The CRW is located in the 
basement of the library and has a computer lab, job search resources, and 
book collections. 

• Seward Park. The Seward Park branch is located on Manhattan’s Lower 
East Side, near Chinatown, but because of ongoing renovations since 2002, 
the program has temporarily moved to the Tompkins Street branch. The 
CRW serves Chinese, Puerto-Rican, African-American, and Afro- 
Caribbean students. Approximately 80 students participate in small-group 
tutoring and study in the computer lab. The CRW also has an extensive col- 
lection of multicultural materials. 
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Oakland Public Library 

Second Start — Oakland Public Library’s adult literacy program — started in 1984 and 
is a small, community-based alternative program with classroom instruction and one-on-one 
tutoring. Housed in a downtown office building, Second Start’s multiethnic staff offer an inten- 
sive and personalized curriculum in math, writing, and spelling as well as instruction in stress 
reduction. The classes are taught by professional instructors; some classes have a fixed sched- 
ule, while others are open-entry, open-exit. Second Start also offers a computer-assisted literacy 
program, with more than 20 computers. The majority of Second Start’s funding comes fix)m the 
City of Oakland, with some additional support from private sources. In the center’s informal 
meeting room, free food donated by neighborhood restaurants is often available. Nearly 150 
volunteers serve as one-on-one tutors both in-house and off-site. Several students have been 
hired as staff, and the program makes efforts to share decision-making with its students. 



Queens Borough Public Library 

Queens Borough Public Library is among the nation’s oldest and largest library systems 
and serves one of the most diverse populations. The adult literacy program, founded in 1977, 
has its roots in earlier library-based programs that provided education to immigrants. Currently, 
6 of the library’s 62 branches support Adult Learning Centers (ALCs), which are funded 
through the New York City Adult Literacy Initiative (NYCALI), other government programs, 
and foundations. The ALCs enroll more than 2,500 adults each year, offering ESOL, pre-GED, 
and basic literacy instruction as well as computer classes, self-study with video- and audiotapes, 
literacy collections, and a student-writing journal entitled The Open Door. Three of the six 
ALCs are included in the LILAA persistence study. 

• Central. The Central branch’s ALC serves about 300 adult students in the 
Jamaica section of Queens. Volunteers lead small-group tutoring. Students 
also use a computer lab that has Internet access and self-study materials. 

• Flushing. The Flushing branch is the largest in the Queens system and en- 
rolls approximately 500 adult literacy students each year. Located in a com- 
mercial district of an area populated by immigrants from all over the world, 
the ALC provides ESOL and basic literacy study to students mainly of Asian 
descent. The program also serves many drop-in clients who are seeking so- 
cial service referrals and resources. 

• Rochdale Village. The Rochdale Village branch houses a smaller program 
that serves about 100 students. It is located in a middle- and working-class 
neighborhood and serves mostly African-American and Afro-Caribbean 
students. The ALC is housed in one room at the back of the library and of- 
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fers small-group instruction, computer access, and a large collection of lit- 
eracy materials. 



Redwood City Public Library 

Project READ, in Redwood City, California, serves approximately 200 adults who have 
low literacy skills. More than 180 volunteer tutors work with these adult students either one-on- 
one or in small groups. The program also has five computers and provides help in using them. 
Two-thirds of the students are Hispanic, and many do not have a high school diploma or GED. 
Project READ serves a variety of other populations, including children and teens in the com- 
munity, people with learning disabilities, and adults in a local prison. The program receives 
fimding through the library’s general fund and receives additional funding and resources fix>m 
individuals, businesses, and foundations. 
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Appendix C 

Issues Related to Capturing Student Persistence 
Using Attendance Record Data 
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Chapter 3 in this report discusses challenges that the Literacy in Libraries Across America 
(LILAA) programs have faced in collecting and managing data for this persistence study and for 
tracking students’ participation patterns. These challenges — related to respecting students’ pri- 
vacy, collecting information for tutors, capturing time spent in independent learning activities, and 
issues centering on technological capacity and accuracy — all have implications for the interpreta- 
tion of the findings presented in this report. In particular. Chapter 2, which presents an analysis of 
students’ characteristics and their persistence, refers readers to this appendix in discussions related 
to the samples of program entrants: the short follow-up sample and the extended follow-up sam- 
ple. This appendix gives readers a better understanding of the challenges related to analyzing the 
quantitative data on students’ characteristics and participation patterns as collected by the LILAA 
programs in this study, and it speaks to the reliability of the statistical analyses presented in this 
report. These analytical challenges are likely to arise in many other literacy and education pro- 
grams, which can learn fiom the LILAA programs’ experiences. 



Missing Data 

Because of the many challenges that the LILAA programs face in collecting and man- 
aging data for the persistence study, it is inevitable that some student participation will not be 
captured in the data provided for the evaluation. Despite such underreporting, however, the data 
can still reveal valuable trends in participation patterns at each program. Because the data for 
this report cover a long period of time (fiom January 2000 to December 2001), periodic lapses 
in entering data on students’ attendance will not invalidate the larger picture of their persistence. 
Also, although data are likely missing, it can usually be assumed that they will be missing at a 
relatively consistent level over time, because the programs tend to be faced with the same set of 
challenges. Therefore, trends over time reveal trends in participation. 

In the fall of 2001, the researchers started to review program data regularly, often on a 
monthly basis. This allows the researchers to analyze trends as they occur, to review the find- 
ings with program staff, and to check whether the findings reveal actual trends or a change in 
the site’s data collection and entry methods. The ongoing review of data also allows program 
staff to see trends in persistence and to consider the site’s interventions in this light. Therefore, 
if the programs improve their data collection, the researchers can take this into account in inter- 
preting the data. 

Finally, because of quality control problems during the early stages of data collection, it 
is possible that overreporting also occurred. 
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Consistency of Measures Across Programs 

Although this report does not make comparisons of trends across programs, it is still use- 
ful to have some consistency across programs, so that the programs can be discussed in the same 
way. For example, it is important to ensure that when a student is termed “active” in a library liter- 
acy program, this means the same thing regardless of which program is involved. Such a defini- 
tion may differ fiom the operational definitions used by the programs themselves. For example, 
staff at some programs may consider a student active if the student has not told a staff member 
that he or she is dropping out or if the student has remained informally connected to the program 
but has not actually attended and participated. Other programs may consider a student active only 
if the student has participated in an activity during a given time period. However, the research 
team needs to find a way to define “active” so that it has the same meaning throughout the study. 
This may result in the researchers’ finding different trends (or numbers of students being served) 
than the programs themselves find. For the purposes of this study, active status is determined by 
the hours entered for students; thus, if a students has any hours, for any activity, entered in the da- 
tabase, that student is considered active for the given time period. 

Similarly, it is important to have common definitions for such terms as “program en- 
try,” “program exit,” “employment,” “goal achievement,” and so on. Throughout this study, the 
researchers have been cognizant that definitions may differ fi-om program to program, and they 
have taken great care either to create a common definition or to make sure that different defini- 
tions are identified as such. Also, some programs may not collect attendance information for all 
the literacy activities they offer. For example, some programs collect data on students’ time 
spent in the library doing personal research and reading, while other programs do not. Although 
this study does not compare programs, it is still important to realize that “participation” can 
mean different things at different sites. 



What the Data Do Not Capture 

The literacy programs in the LILAA persistence study have set up their database sys- 
tems to collect participation data for some or all of the literacy activities they offer — including 
class and group instruction, pair-tutoring, individual activities, and sometimes such “extracur- 
ricular” activities such as field trips, art programs, and family events. However, in addition to 
the challenges of collecting attendance data for all these activities, there are a number of other 
literacy activities that program staff cannot quantify and include in any of the data. For example, 
students’ time spent doing homework, reading on their own at home, or engaging in conversa- 
tion with their classmates is very important to success in achieving literacy, but no data reflect 
such activities, which are also a key part of persistence in literacy learning. However, it is as- 
sumed that students enroll in the library literacy programs because they can benefit fi-om the 
assistance and support that these programs offer. Therefore, analyzing persistence in participa- 
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tion at the library programs alone is very valuable. Moreover, many of the literacy activities that 
students pursue on their own time are often a result of participating in a literacy program. 

In addition to this study’s limitations in measuring individual literacy activities, its quanti- 
tative analyses (and the programs’ databases) are not designed to capture many other aspects of 
students’ lives that are relevant to their literacy experiences. For example, the data do not track 
students’ employment histories after intake; they do not follow outcomes for their families and 
children; and they do not capture students’ educational progress after they leave the literacy pro- 
grams (with the exception of students who are included in the achievement study). The research- 
ers also do not consistently have information on the educational attainment level of students. Most 
important, in most cases they do not have data on why students leave library literacy programs. 
Thus, the assessment of both the programs and the students they serve is necessarily limited. 

In sum, collecting and managing data on who students are and just how they are spend- 
ing their time in literacy programs is a substantial challenge for libraries. It requires a lot of staff 
time to implement processes for tracking attendance; to follow up with instructors, tutors, and 
students; to set up and learn new software; and to enter large amounts of data into the system. 
And with the added challenges of privacy concerns and noncompliance of tutors and students, 
there is always a large possibility that not all data are being recorded. However, despite missing 
information and other limitations, the data collected for this study allow for valuable and valid 
analyses of student persistence. These data may not suffice for giving accurate counts of the 
total number of students participating in each of the LILAA programs, but analyzing the data in 
the aggregate and over a long period of time allows for analyses of individual and program 
trends in participation that reveal important observations about persistence. 
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Appendix D 



The Achievement Study Component of the 
LILAA Persistence Study 



The Design of the Achievement Study 

Within the larger context of the Literacy in Libraries Across America (LILAA) 
persistence study, the achievement study component was designed to provide the opportunity to 
learn about: 

• The literacy level of students in the LILAA programs 

• How achievement improves as students spend more time in the programs 

• How achievement progresses after students leave the programs 

The design involves measuring student achievement at two points in time, 
approximately 12 months apart, with a battery of literacy tests. The literacy levels reported in 
Chapter 2 of this report are based on findings from the first point in time. To the extent possible, 
all the students who participated in the first wave of the tests have been tracked and retested, 
regardless of whether they remained active in their LILAA program. The results of these tests 
are being supplemented with in-depth qualitative interviews of a subsample of these students, to 
capture (1) the extent of participation in literacy activities inside and outside the library literacy 
programs and (2) students' perceptions of changes in their literacy levels. 



The Tests Used in the Achievement Study 

The battery of tests selected for the achievement study consists of instruments that are 
all considered program-based and learner-centered, and they all focus on the learning process as 
program outcomes. Most important, these tests rely on different procedures for different 
students, making them more appropriate for each individual and thus more meaningful and 
valid. These tests are reputable, standardized tests that are used nationally, which allows for 
library literacy programs to be assessed among other adult education providers, giving them an 
equal base of comparison. This is the first time that a battery of standardized tests has been 
given specifically to a cohort of library literacy programs within a systematic study of them. 
The tests are described below. 

1. The Test of Word Reading Efficiency (TOWRE), published by Pro-Ed, 
measures reading rate and word recognition. Reading rate and word 
recognition are important predictors of reading comprehension. (Someone 
who reads too slowly loses the meaning of long and complicated sentences.) 

The TOWRE consists of two subtests: 
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o The Sight Word Efficiency (SWE) subtest measures the number of 
printed words that can be correctly identified within 45 seconds. 

• The Phonemic Decoding Efficiency (PDE) subtest measures the 
number of pronounceable words that the test-taker can decode in 45 
seconds.' 

2. The Peabody Picture Vocabulary Test (PPVT) measures vocabulary 
skills, assessing both verbal and auditory attainment of Standard English. It 
is a measure of listening and reading vocabulary. The test can be 
administered to persons of any age. Test-takers are asked to select pictures 
that best match the meaning of words that are read aloud by the person 
administering the test.^ 

3. The Adult Basic Learning Examination (ABLE) test measures several 
skills, including reading comprehension (only the reading comprehension 
part is used in this study). For the reading comprehension subtest, test-takers 
are presented with signs and short reading passages about the day-to-day 
lives of adults. The passages are followed by questions that test 
comprehension of the text and the ability to make inferences. The test has 
three levels: Level 1 is for adults with one to four years of education (primary 
schooling); Level 2 is for adults with five to eight years of schooling 
(intermediate schooling); and Level 3 is for adults who have had at least 
eight years of schooling but who have not graduated fi'om high school.^ 

4. The Basic English Skills Test (BEST) is a special test for students of 
English as a Second Language (ESL); it measures English speaking and 
listening skills. It is designed to measure competency-based listening 
comprehension, speaking, and elementary reading and wnting skills. Test- 
takers are presented with a series of real-life listening and speaking tasks, 
such as telling time, paying for a store item, and giving and receiving 
directions.'* Only those sample members who are learners of English (57 
students) took the BEST. See Appendix Table D.l. 

Though the achievement tests reported in Figure 2.2 were selected to measure 
different literacy outcomes, some of the differences in achievement levels across the tests are 



'Torgesen, Wagner, and Rashotte, 1999. 
^Dunn and Dunn, 1997. 

^Karlsen and Gardner, 1986. 

■’Center for Applied Linguistics, 1989. 
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notable. The most highly correlated tests are the ABLE and the TOWRE,’ as would be 
expected, because the rate of word recognition (as measured by the TOWRE) is an important 
predictor of reading comprehension (as measured by the ABLE). The absence of correlation 
between the PPVT and the other tests may suggest that the vocabulary skills measured by the 
PPVT are distinct and surprisingly unrelated to the other literacy skills as measured with this 
battery of tests. 



^There is a statistically significant correlation coefficient of .5 1 between the ABLE and the TOWRE 
SWE subtest, and there is a statistically significant correlation coefficient of .45 between the ABLE and 
the TOWRE PDE subtest. 
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Recent Publications on MDRC Projects 



Note: For works not published by MDRC, the publisher's name is shown in parentheses. With a few exceptions, this 
list includes reports published by MDRC since 1999. A complete publications list is available from MDRC 
and on its Web site (www.mdrc.org), from which copies of MDRC's publications can also be downloaded. 



Reforming Welfare and Making 
Work Pay 

Next Generation Project 
A collaboration among researchers at MDRC and 
several other leading research institutions focused on 
studying the effects of welfare, antipoverty, and 
enployment policies on children and families. 

How Welfare and Work Policies Affect Children: A 
Synthesis of Research. 2001. Pamela Morris, 

Aletha Huston, Greg Duncan, Danielle Crosby, 
Johannes Bos. 

How Welfare and Work Policies Affect Employment 
and Income: A Synthesis of Research. 2001. Dan 
Bloom, Charles Michalopoulos. 

How Welfare and Work Policies for Parents Affect 
Adolescents: A Synthesis of Research. 2002. Lisa 
Gennetian, Greg Duncan, Virginia Knox, Wanda 
Vargas, Elizabeth Clark-Kaufftnan, Andrew 
London. 

ReWORKing Welfare: Technical Assistance 
for States and Localities 

A multifaceted effort to assist states and localities in 
designing and implementing their welfare reform 
programs. The project includes a series of “how-to” 
guides, conferences, briefings, and customized, in- 
depth technical assistance. 

After AFDC: Welfare-to- Work Choices and 
Challenges for States. 1997. Dan Bloom. 

Work First: How to Implement an Employment- 
Focused Approach to Welfare Reform. 1997. Amy 
Brown. 

Business Partnerships: How to Involve Employers in 
Welfare Reform. 1998. Amy Brown, Maria Buck, 
Erik Skinner. 

Promoting Participation: How to Increase 
Involvement in Welfare-to- Work Activities. 1999. 
Gayle Hamilton, Susan Scrivener. 

Encouraging Work, Reducing Poverty: The Impact of 
Work Incentive Programs. 2000. Gordon Berlin. 
Steady Work and Better Jobs: How to Help Low- 
Income Parents Sustain Employment and Advance 
in the Workforce. 2000. Julie Strawn, Karin 
Martinson. 

Beyond Work First: How to Help Hard-to-Employ 
Individuals Get Jobs and Succeed in the 
Workforce. 200 1 . Amy Brown. 







Project on Devolution and Urban Change 

A multiyear study in four major urban counties — 
Cuyahoga County, Ohio (which includes the city of 
Cleveland), Los Angeles, Miami-Dade, and 
Philadelphia — that examines how welfare reforms 
are being implemented and affect poor people, their 
neighborhoods, and the institutions that serve them. 

Big Cities and Welfare Reform: Early 
Implementation and Ethnographic Findings from 
the Project on Devolution and Urban Change. 
1999. Janet Quint, Kathryn Edin, Maria Buck, 
Barbara Fink, Yolanda Padilla, Olis Simmons- 
Hewitt, Mary Valmont. 

Food Security and Hunger in Poor, Mother-Headed 
Families in Four U.S. Cities. 2000. Denise Polit, 
Andrew London, John Martinez. 

Assessing the Impact of Welfare Reform on Urban 
Communities: The Urban Change Project and 
Methodological Considerations. 2000. Charles 
Michalopoulos, Johannes Bos, Robert Lalonde, 
Nandita Verma. 

Post-TANF Food Stamp and Medicaid Benefits: 
Factors That Aid or Impede Their Receipt. 2001. 
Janet Quint, Rebecca Widom. 

Social Service Organizations and Welfare Reform. 

2001. Barbara Fink, Rebecca Widom. 

Monitoring Outcomes for Cuyahoga County 's 
Welfare Leavers: How Are They Faring? 2001. 
Nandita Verma, Claudia Coulton. 

The Health of Poor Urban Women: Findings from 
the Project on Devolution and Urban Change. 
2001. Denise Polit, Andrew London, John 
Martinez. 

Is Work Enough? The Experiences of Current and 
Former Welfare Mothers Who Work. 2001. Denise 
Polit, Rebecca Widom, Kathryn Edin, Stan Bowie, 
Andrew London, Ellen Scott, Abel Valenzuela. 
Readying Welfare Recipients for Work: Lessons from 
Four Big Cities as They Implement Welfare 
Reform. 2002. Thomas Brock, Laura Nelson, 
Megan Reiter. 

Welfare Reform in Cleveland: Implementation, 
Effects, and Experiences of Poor Families and 
Neighborhoods. 2002. Thomas Brock, Claudia 
Coulton, Andrew London, Denise Polit, Lashawn 
Richburg-Hayes, Ellen Scott, Nandita Verma. 




Comparing Outcomes for Los Angeles County's 
HVD~ Assisted and Unassisted CalWORKs 
Leavers. 2003. Nandita Verma, Richard Hendra. 
Monitoring Outcomes for Los Angeles County 's Pre- 
and Post-CalWORKs Leavers: How Are They 
Faring? 2003. Nandita Verma, Richard Hendra. 

Wisconsin Works 

This study examines how Wisconsin’s welfare-to- 
work program, one of the first to end welfare as an 
entitlement, is administered in Milwaukee. 

Complaint Resolution in the Context of Welfare 
Reform: How W-2 Settles Disputes. 2001. Suzanne 
Lynn. 

Exceptions to the Rule: The Implementation of 24- 
Month Time-Limit Extensions in W-2. 2001. Susan 
Gooden, Fred Doolittle. 

Matching Applicants with Services: Initial 
Assessments in the Milwaukee County W~2 
Program. 2001. Susan Gooden, Fred Doolittle, Ben 
Glispie. 

Employment Retention and Advancement 
Project 

Conceived and funded by the U.S. Department of 
Health and Human Services (HHS), this demon- 
stration project is aimed at testing various ways to 
help low-income people find, keep, and advance in 
jobs. 

New Strategies to Promote Stable Employment and 
Career Progression: An Introduction to the 
Employment Retention and Advancement Project 
(HHS). 2002. Dan Bloom, Jacquelyn Anderson, 
Melissa Wavelet, Karen Gardiner, Michael 
Fishman. 

Time Limits 

Welfare Time Limits: State Policies, Implementation, 
and Effects on Families. 2002. Dan Bloom, Mary 
Farrell, Barbara Fink. 

Leavers, Stayers, and Cyclers: An Analysis of the 
Welfare Caseload. 2002. Cynthia Miller. 

Florida’s Family Transition Program 

An evaluation of Florida’s initial time-limited welfare 
program, which includes services, requirements, and 
financial work incentives intended to reduce long- 
term welfare receipt and help welfare recipients find 
and keep jobs. 

The Family Transition Program: Implementation and 
Three-Year Impacts of Florida 's Initial Time- 
Limited Welfare Program. 1999. Dan Bloom, Mary 
Farrell, James Kemple, Nandita Verma. 

The Family Transition Program: Final Report on 
Florida 's Initial Time-Limited Welfare Program. 
2000. Dan Bloom, James Kemple, Pamela Morris, 
Susan Scrivener, Nandita Verma, Richard Hendra. 



Cross-State Study of Time-Limited Welfare 

An examination of the implementation of some of the 
first state-initiated time-limited welfare programs. 

Welfare Time Limits: An Interim Report Card. 1999. 
Dan Bloom. 

Connecticut’s Jobs First Program 

An evaluation of Connecticut’s statewide time-limited 
welfare program, which includes financial work 
incentives and requirements to participate in 
employment-related services aimed at rapid job 
placement. This study provides some of the earliest 
information on the effects of time limits in major 
urban areas. 

Connecticut Post-Time Limit Tracking Study: Six- 
Month Survey Results. 1999. Jo Anna Hunter- 
Manns, Dan Bloom. 

Jobs First: Implementation and Early Impacts of 
Connecticut s Welfare Reform Initiative. 2000. Dan 
Bloom, Laura Melton, Charles Michalopoulos, 
Susan Scrivener, Johanna Walter. 

Connecticut ’s Jobs First Program: An Analysis of 
Welfare Leavers. 2000. Laura Melton, Dan Bloom. 
Final Report on Connecticut 's Welfare Reform 
Initiative. 2002. Dan Bloom, Susan Scrivener, 
Charles Michalopoulos, Pamela Morris, Richard 
Hendra, Diana Adams-Ciardullo, Johanna Walter. 

Vermont’s Welfare Restructuring Project 

An evaluation of Vermont’s statewide welfare reform 
program, which includes a work requirement after a 
certain period of welfare receipt, and financial work 
incentives. 

Forty-Two-Month Impacts of Vermont's Welfare 
Restructuring Project. 1999. Richard Hendra, 
Charles Michalopoulos. 

WRP: Key Findings from the Forty-Two-Month 
Client Survey. 2000. Dan Bloom, Richard Hendra, 
Charles Michalopoulos. 

WRP: Final Report on Vermont 's Welfare 
Restructuring Project. 2002. Susan Scrivener, 
Richard Hendra, Cindy Redcross, Dan Bloom, 
Charles Michalopoulos, Johanna Walter. 

Financial Incentives 

Encouraging Work, Reducing Poverty: The Impact of 
Work Incentive Programs. 2000. Gordon Berlin. 

Minnesota Family Investment Program 

An evaluation of Minnesota’s pilot welfare reform 
initiative, which aims to encourage work, alleviate 
poverty, and reduce welfare dependence. 

Reforming Welfare and Rewarding Work: Final 
Report on the Minnesota Family Investment 
Program. 2000; 




Volume 1: Effects on Adults. Cynthia Miller, 
Virginia Knox, Lisa Gennetian, Martey Dodoo, Jo 
Anna Hunter, Cindy Redcross. 

Volume 2: Effects on Children. Lisa Gennetian, 
Cynthia Miller. 

Reforming Welfare and Rewarding Work: A 
Summary of the Final Report on the Minnesota 
Family Investment Program. 2000. Virginia Knox, 
CyntWa Miller, Lisa Gennetian. 

Final Report on the Implementation and Impacts of 
the Minnesota Family Investment Program in 
Ramsey County. 2000. Patricia Auspos, Cynthia 
Miller, Jo Anna Hunter. 

New Hope Project 

A test of a community-based, work-focused antipoverty 
program and welfare alternative operating in 
Milwaukee. 

New Hope for People with Low Incomes: Two-Year 
Results of a Program to Reduce Poverty and 
Reform Welfare. 1999. Johannes Bos, Aletha 
Huston, Robert Granger, Greg Duncan, Thomas 
Brock, Vonnie McLoyd. 

Canada’s Self-Sufficiency Project 

A test of the effectiveness of a temporary earnings 
supplement on the employment and welfare receipt of 
public assistance recipients. Reports on the Self- 
Sufficiency Project are available from: Social 
Research and Demonstration Corporation (SRDC), 
275 Slater St., Suite 900, Ottawa, Ontario KIP 5H9, 
Canada. Tel.: 613-237-4311; Fax: 613-237-5045. In 
the United States, the reports are also available from 
MDRC. 

Does SSP Plus Increase Employment? The Effect of 
Adding Services to the Self-Sufficiency Project 's 
Financial Incentives (SRDC). 1999. Gail Quets, 
Philip Robins, Elsie Pan, Charles Michalopoulos, 
David Card. 

When Financial Work Incentives Pay for Themselves: 
Early Findings from the Self-Sufficiency Project 's 
Applicant Study (SRDC). 1999. Charles 
Michalopoulos, Philip Robins, David Card. 

The Self-Sufficiency Project at 36 Months: Effects of 
a Financial Work Incentive on Employment and 
Income (SRDC). 2000. Charles Michalopoulos, 
David Card, Lisa Gennetian, Kristen Harknett, 
Philip K. Robins. 

The Self-Sufficiency Project at 36 Months: Effects on 
Children of a Program That Increased Parental 
Employment and Income {SRI3C). 2000. Pamela 
Morris, Charles Michalopoulos. 

When Financial Incentives Pay for Themselves: 
Interim Findings from the Self-Sufficiency 
Project's Applicant Study (SRDC). 2001. Charles 
Michalopoulos, Tracey Hoy. 



SSP Plus at 36 Months: Effects of Adding 
Employment Services to Financial Work Incentives 
(SRDC). 2001. Ying Lei, Charles Michalopoulos. 
Making Work Pay: Final Report on the Self- 
Sufficiency Project for Long-Term Welfare 
Recipients (SRDC). 2002. Charles Michalopoulos, 
Doug Tattrie, Cynthia Miller, Philip Robins, 

Pamela Morris, David Gyarmati, Cindy Redcross, 
Kelly Foley, Reuben Ford. 

Mandatory Welfare Employment Programs 

National Evaluation of Welfare-to-Work 
Strategies 

Conceived and sponsored by the U.S. Department of 
Health and Human Services (HHS), with support 
from the U.S. Department of Education (ED), this is 
the largest-scale evaluation ever conducted of 
different strategies for moving people from welfare to 
employment. 

Do Mandatory Welfare-to-Work Programs Affect the 
Well-Being of Children? A Synthesis of Child 
Research Conducted as Part of the National 
Evaluation of Welfare-to- Work Strategies 
(HHS/ED). 2000. Gayle Hamilton. 

Evaluating Alternative Welfare-to- Work Approaches: 
Two-Year Impacts for Eleven Programs 
(HHS/ED). 2000. Stephen Freedman, Daniel 
Friedlander, Gayle Hamilton, JoAnn Rock, Marisa 
Mitchell, Jodi Nudelman, Amanda Schweder, 

Laura Storto. 

Impacts on Young Children and Their Families Two 
Years After Enrollment: Findings from the Child 
Outcomes Study (HHS/ED). 2000. Sharon 
McGroder, Maatha Zaslow, Kristin Moore, Suzanne 
LeMenestrel. 

What Works Best for Whom: Impacts of 20 Welfare-to- 
Work Programs by Subgroup (HHS/ED). 2000. 
Charles Michalopoulos, Christine Schwartz. 
Evaluating Two Approaches to Case Management: 
Implementation, Participation Patterns, Costs, and 
Three-Year Impacts of the Columbus Welfare-to- 
Work Program (HHS/ED). 2001. Susan Scrivener, 
Johanna Walter. 

How Effective Are Different Welfare-to-Work 
Approaches? Five-Year Adult and Child Impacts for 
Eleven Programs - Executive Summary (HHS/ED). 
2001. Gayle Hamilton, Stephen Freedman, Lisa 
Gennetian, Charles Michalopoulos, Johanna Walter, 
Diana Adams-Ciardullo, Anna Gassman-Pines, Sharon 
McGroder, Martha Zaslow, Suijeet Ahluwalia, 

Jennifer Brooks. 

Moving People from Welfare to Work: Lessons from 
the National Evaluation of Welfare-to- Work 
Strategies (HHS/ED). 2002. Gayle Hamilton. 
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Los Angeles’s Jobs-First GAIN Program 
An evaluation of Los Angeles’s refocused GAIN 
(welfare-to-work) program, which emphasizes rapid 
employment. This is the first in-depth study of a full- 
scale “work first” program in one of the nation’s 
largest urban areas. 

The Los Angeles Jobs-First GAIN Evaluation: First- 
Year Findings on Participation Patterns and 
Impacts. 1999. Stephen Freedman, Marisa 
Mitchell, David Navarro. 

The Los Angeles Jobs-First GAIN Evaluation: Final 
Report on a Work First Program in a Major Urban 
Center. 2000. Stephen Freedman, Jean Knab, Lisa 
Gennetian, David Navarro. 

Teen Parents on Welfare 

Teenage Parent Programs: A Synthesis of the Long- 
Term Effects of the New Chance Demonstration, 
Ohio 's Learning, Earning, and Parenting (LEAP) 
Program, and the Teenage Parent Demonstration 
(TPD). 1998. Robert Granger, Rachel Cytron. 

Ohio’s LEAP Program 

An evaluation of Ohio’s Learning, Earning, and 
Parenting (LEAP) Program, which uses financial 
incentives to encourage teenage parents on welfare to 
stay in or return to school. 

LEAP: Final Report on Ohio 's Welfare Initiative to 
Improve School Attendance Among Teenage 
Parents. 1997. Johannes Bos, Veronica Fellerath. 

New Chance Demonstration 

A test of a comprehensive program of services that 
seeks to inprove the economic status and general 
well-being of a group of highly disadvantaged young 
women and their children. 

New Chance: Final Report on a Comprehensive 
Program for Young Mothers in Poverty and Their 
Children. 1997. Janet Quint, Johannes Bos, Denise 
Polit. 

Parenting Behavior in a Sample of Young Mothers in 
Poverty: Results of the New Chance Observational 
Study. 1998. Martha Zaslow, Carolyn Eldred, 
editors. 

Center for Employment Training 
Replication 

This study is testing whether the successful results for 
youth of a training program developed in San Jose 
can be replicated in 12 other sites around the country. 



Evaluation of the Center for Employment Training 
Replication Sites: Interim Report (Berkeley Policy 
Associates). 2000. Stephen Walsh, Deana 
Goldsmith, Yasuyo Abe, Andrea Cann. 

Focusing on Fathers 

Parents^ Fair Share Demonstration 
A demonstration for unemployed noncustodial 
parents (usually fathers) of children on welfare. PFS 
aims to improve the men’s employment and earnings, 
reduce child poverty by increasing child support 
payments, and assist the fathers in playing a broader 
constructive role in their children’s lives. 

Fathers ' Fair Share: Helping Poor Men Manage 
Child Support and Fatherhood (Russell Sage 
Foundation). 1999. Earl Johnson, Ann Levine, Fred 
Doolittle. 

Parenting and Providing: The Impact of Parents ' 
Fair Share on Paternal Involvement. 2000. 

Virginia Knox, Cindy Redcross. 

Working and Earning: The Impact of Parents ' Fair 
Share on Low-Income Fathers ' Employment. 2000. 
JohnM. Martinez, Cynthia Miller. 

The Responsible Fatherhood Curriculum. 2000. 

Eileen Hayes, with Kay Sherwood. 

77ie Challenge of Helping Low-Income Fathers 
Support Their Children: Final Lessons from 
Parents ' Fair Share. 200 1 . Cynthia Miller, 

Virginia Knox. 

Career Advancement and Wage 
Progression 

Opening Doors to Earning Credentials 
An exploration of strategies for increasing low-wage 
workers’ access to and completion of community 
college programs. 

Opening Doors: Expanding Educational Oppor- 
tunities for Low-Income Workers. 2001. Susan 
Golonka, Lisa Matus-Grossman. 

Welfare Reform and Community Colleges: A Policy 
and Research Context. 2002. Thomas Brock, Lisa 
Matus-Grossman, Gayle Hamilton. 

Opening Doors: Students ' Perspectives on Juggling 
Work, Family, and College. 2002. Lisa Matus- 
Grossman, Susan Gooden. 

Opening Doors: Supporting CalWORKs Students at 
California Community Colleges: An Exploratory 
Focus Group Study. 2002. Laura Nelson, Rogeair 
Purnell. 



EducatioE Reform 

Career Academies 

The largest and most comprehensive evaluation of a 
school-to-work initiative, this study examines a 
promising approach to high school restructuring and 
the school-to-work transition. 

Career Academies: Building Career Awareness and 
Work-Based Learning Activities Through Employer 
Partnerships. 1999. James Kemple, Susan 
Poglinco, Jason Snipes. 

Career Academies: Impacts on Students ' 

Engagement and Performance in High School. 

2000. James Ken^le, Jason Snipes. 

Career Academies: Impacts on Students ' Initial 

Transitions to Post-Secondary Education and 
Employment. 2001. James Kemple. 

First Things First 

This demonstration and research project looks at First 
Things First, a whole-school reform that combines a 
variety of best practices aimed at raising achievement 
and graduation rates in both urban and mral settings. 

Scaling Up First Things First: Site Selection and the 
Planning Year. 2002. Janet Quint. 

Closing Achievement Gaps 
Conducted for the Council of the Great City Schools, 
this study identifies districtwide approaches to urban 
school reform that appear to raise overall student 
performance while reducing achievement gaps among 
racial groups. 

Foundations for Success: Case Studies of How 
Urban School Systems Improve Student 
Achievement. 2002. Jason Snipes, Fred Doolittle, 
Corinne Herlihy. 

Project GRAD 

This evaluation examines Project GRAD, an 
education initiative targeted at urban schools and 
combining a number of proven or promising reforms. 

Building the Foundation for Improved Student 
Performance: The Pre-Curricular Phase of Project 
GRAD Newark. 2000. Sandra Ham, Fred Doolittle, 
Glee Ivory Holton. 

Accelerated Schools 

This study examines the inplementation and inpacts 
on achievement of the Accelerated Schools model, a 
whole-school reform targeted at at-risk students. 

Evaluating the Accelerated Schools Approach: A 
Look at Early Implementation and Impacts on 
Student Achievement in Eight Elementary Schools. 

2001. Howard Bloom, Sandra Ham, Laura Melton, 
Julienne O’Brien. 



Extended-Service Schools Initiative 
Conducted in partnership with Public/Private 
Ventures (P/PV), this evaluation of after-school 
programs operated as part of the Extended-Service 
Schools Initiative examines the programs’ implemen- 
tation, quality, cost, and effects on students. 

Multiple Choices After School: Findings from the 
Extended-Service Schools Initiative (P/PV). 2002. 
Jean Baldwin Grossman, Marilyn Price, Veronica 
Fellerath, Linda Jucovy, Lauren Kotloff, Rebecca 
Raley, Karen Walker. 

School-to-Work Project 

A study of innovative programs that help students 
make the transition from school to work or careers. 

Home-Grown Lessons: Innovative Programs Linking 
School and Work (Jossey-Bass Publishers). 1995. 
Edward Pauly, Hilary Kopp, Joshua Haimson. 
Home-Grown Progress: The Evolution of Innovative 
School-to- Work Programs. 1997. Rachel Pedraza, 
Edward Pauly, Hilary Kopp. 

Project Transition 

A demonstration program that tested a combination of 
school-based strategies to facilitate students’ 
transition from middle school to high school. 

Project Transition: Testing an Intervention to Help 
High School Freshmen Succeed. 1999. Janet Quint, 
Cynthia Miller, Jennifer Pastor, Rachel Cytron. 

Equity 2000 

Equity 2000 is a nationwide initiative sponsored by 
the College Board to improve low-income students’ 
access to college. The MDRC paper examines the 
implementation of Equity 2000 in Milwaukee Public 
Schools. 

Getting to the Right Algebra: The Equity 2000 
Initiative in Milwaukee Public Schools. 1999. 
Sandra Ham, Erica Walker. 

Employment and Community 
Initiatives 

Jobs-Plus Initiative 

A multisite effort to greatly increase employment 
among public housing residents. 

Mobilizing Public Housing Communities for Work: 
Origins and Early Accomplishments of the Jobs- 
Plus Demonstration. 1999. James Riccio. 

Building a Convincing Test of a Public Housing 
Employment Program Using Non-Experimental 
Methods: Planning for the Jobs-Plus 
Demonstration. 1999. Howard Bloom. 
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Jobs-Plus Site-by-Site: An Early Look at Program 
Implementation. 2000. Edited by Susan Philipson 
Bloom with Susan Blank. 

Building New Partnerships for Employment: 
Collaboration Among Agencies and Public 
Housing Residents in the Jobs-Plus Demonstration. 

2001. Linda Kato, James Riccio. 

Making Work Pay for Public Housing Residents: 
Financial-Incentive Designs at Six Jobs- Plus 
Demonstration Sites. 2002. Cynthia Miller, James 
Riccio. 

The Special Challenges of Offering Employment 
Programs in Culturally Diverse Communities: The 
Jobs-Plus Experience in Public Housing 
Developments. 2002. Linda Kato. 

The Employment Experiences of Public Housing 
Residents: Findings from the Jobs-Plus Baseline 
Survey. 2002. John Martinez. 

Children in Public Housing Developments: An 
Examination of the Children at the Beginning of 
the Jobs-Plus Demonstration. 2002. Pamela 
Morris, Stephanie Jones. 

Jobs-Plus Site-by-Site: Key Features of Mature 
Employment Programs in Seven Public Housing 
Communities. 2003. Linda Kato. 

Neighborhood Jobs Initiative 

An initiative to increase employment in a number of 
low-income communities. 

The Neighborhood Jobs Initiative: An Early Report 
on the Vision and Challenges of Bringing an 
Employment Focus to a Community-Building 
Initiative. 2001. Frieda Molina, Laura Nelson. 
Structures of Opportunity: Developing the 
Neighborhood Jobs Initiative in Fort Worth, Texas. 

2002. TonyProscio. 

Connections to Work Project 

A study of local efforts to increase competition in the 
choice of providers of employment services for 
welfare recipients and other low-income populations. 
The project also provides assistance to cutting-edge 
local initiatives aimed at helping such people access 
and secure Jobs. 

Designing and Administering a Wage-Paying 
Community Service Employment Program Under 
TANF: Some Considerations and Choices. 1999. 
Kay Sherwood. 

San Francisco Works: Toward an Employer-Led 
Approach to Welfare Reform and Workforce 
Development. 2000. Steven Bliss. 



Canada’s Earnings Supplement Project 
A test of an innovative financial incentive intended to 
expedite the reemployment of displaced workers and 
encourage full-year work by seasonal or part-year 
workers, thereby also reducing receipt of 
unemployment insurance. 

Testing a Re-Employment Incentive for Displaced 
Workers: The Earnings Supplement Project. 1999. 
Howard Bloom, Saul Schwartz, Susanna Lui-Gurr, 
Suk-Won Lee. 

MDRC Working Papers on 
Research Methodology 

A new series of papers that explore alternative 
methods of examining the implementation and 
inpacts of programs and policies. 

Building a Convincing Test of a Public Housing 
Employ m en t Program Using Non-Experim en tal 
Methods: Planning for the Jobs-Plus 
Demonstration. 1999. Howard Bloom. 

Estimating Program Impacts on Student Achievement 
Using ” Short** Interrupted Time Series. 1999. 
Howard Bloom. 

Using Cluster Random Assignment to Measure 
Program Impacts: Statistical Implications for the 
Evaluation of Education Programs. 1999. Howard 
Bloom, Johannes Bos, Suk-Won Lee. 

The Politics of Random Assignment: Implementing 
Studies and Impacting Policy. 2000. Judith Gueron. 
Assessing the Impact of Welfare Reform on Urban 
Communities: The Urban Change Project and 
Methodological Considerations. 2000. Charles 
Michalopoulos, Joannes Bos, Robert Lalonde, 
Nandita Verma. 

Measuring the Impacts of Whole School Reforms: 
Methodological Lessons from an Evaluation of 
Accelerated Schools. 2001. Howard Bloom. 

A Meta-Analysis of Government Sponsored Training 
Programs. 2001. David Greenberg, Charles 
Michalopoulos, Philip Robins. 

Modeling the Performance of Welfare-to- Work 
Programs: The Effects of Program Management 
and Services, Economic Environment, and Client 
Characteristics. 2001. Howard Bloom, Carolyn 
Hill, James Riccio. 

A Regression- Based Strategy for Defining Subgroups 
in a Social Experiment. 2001. James Kemple, Jason 
Snipes. 





Explaining Variation in the Effects of Welfare-to- 
Work Programs, 2001. David Greenberg, Robert 
Meyer, Charles Michalopoulos, Michael Wiseman. 

Extending the Reach of Randomized Social 
Experiments: New Directions in Evaluations of 
American Welfare-to-Work and Employment 
Initiatives, 2001. James Riccio, Howard Bloom. 

Can Nonexperimental Comparison Group Methods 
Match the Findings from a Random Assignment 
Evaluation of Mandatory Welfare-to-Work 
Programs? 2002. Howard Bloom, Charles 
Michalopoulos, Carolyn Hill, Ying Lei. 



Using Instrumental Variables Analysis to Learn 
More from Social Policy Experiments, 2002. Lisa 
Gennetian, Johannes Bos, Pamela Morris. 

Using Place-Based Random Assignment and 
Comparative Interrupted Time-Series Analysis to 
Evaluate the Jobs-Plus Employment Program for 
Public Housing Residents, 2002. Howard Bloom, 
James Riccio. 
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About MDRC 

The Manpower Demonstration Research Corporation (MDRC) is a nonprofit, 
nonpartisan social policy research organization. We are dedicated to learning what 
works to improve the well-being of low-income people. Through our research and 
the active communication of our findings, we seek to enhance the effectiveness of 
social policies and programs. MDRC was founded in 1974 and is located in New 
York City and Oakland, California. 

MDRC’s current projects focus on welfare and economic security, education, and 
employment and community initiatives. Complementing our evaluations of a wide 
range of welfare reforms are new studies of supports for the working poor and 
emerging analyses of how programs affect children’s development and their 
families’ well-being. In the field of education, we are testing reforms aimed at 
improving the performance of public schools, especially in urban areas. Finally, our 
community projects are using innovative approaches to increase employment in 
low-income neighborhoods. 

Our projects are a mix of demonstrations — field tests of promising program 
models — and evaluations of government and community initiatives, and we 
employ a wide range of methods to determine a program’s effects, including large- 
scale studies, surveys, case studies, and ethnographies of individuals and families. 
We share the findings and lessons Irom our work — including best practices for 
program operators — with a broad audience within the policy and practitioner 
community, as well as the general public and the media. 

Over the past quarter century, MDRC has worked in almost every state, all of the 
nation’s largest cities, and Canada. We conduct our projects in partnership with state 
and local governments, the federal government, public school systems, community 
organizations, and numerous private philanthropies. 
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16 East 34th Street 
New York, NY 10016 
(212) 532-3200 

www.mdrc.org 

475 14th Street 
Oakland, CA 94612 
(510) 663-6372 
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